Add support for MediaStream video track rendering in GPUProcess
authoryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 16 Jan 2020 18:09:09 +0000 (18:09 +0000)
committeryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 16 Jan 2020 18:09:09 +0000 (18:09 +0000)
https://bugs.webkit.org/show_bug.cgi?id=206286

Reviewed by Eric Carlson.

Source/WebCore:

Add a way for WebKit layer to create its own specific SampleBufferDisplayLayer.
WebKit layer can set a function pointer that will be used to create SampleBufferDisplayLayer.
If none is provided, LocalSampleBufferDisplayLayer is used.

Update MediaPlayerPrivateMediaStreamAVFObjC to monitor itself its root layer bounds change.
Conversely, LocalSampleBufferDisplayLayer no longer monitors its root layer bounds change.
This removes the need for SampleBufferDisplayLayer::Client to be notified of bounds change.
We add an API to update the root layer bounds of a LocalSampleBufferDisplayLayer as well,
this API being used by GPUProcess to mirror changes done to WebProcess root layer.

WebRTC incoming video tracks do not have presentation timestamps so we mark them as kCMSampleAttachmentKey_DisplayImmediately.
This is currently lost when doing IPC from WebProcess to NetworkProcess. Add this information for every enqueued sample in LocalSampleBufferDisplayLayer.
We might want in the future to serialize all attachments of the sample.

Manually tested.

* SourcesCocoa.txt:
* WebCore.xcodeproj/project.pbxproj:
* platform/graphics/avfoundation/SampleBufferDisplayLayer.cpp: Added.
(WebCore::SampleBufferDisplayLayer::setCreator):
(WebCore::SampleBufferDisplayLayer::create):
* platform/graphics/avfoundation/SampleBufferDisplayLayer.h:
* platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.h:
* platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.mm:
(-[WebAVSampleBufferStatusChangeListener invalidate]):
(-[WebAVSampleBufferStatusChangeListener start]):
(-[WebAVSampleBufferStatusChangeListener stop]):
(-[WebAVSampleBufferStatusChangeListener observeValueForKeyPath:ofObject:change:context:]):
(WebCore::LocalSampleBufferDisplayLayer::LocalSampleBufferDisplayLayer):
(WebCore::LocalSampleBufferDisplayLayer::~LocalSampleBufferDisplayLayer):
(WebCore::LocalSampleBufferDisplayLayer::updateBoundsAndPosition):
(WebCore::LocalSampleBufferDisplayLayer::enqueueSample):
* platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
* platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
(-[WebRootSampleBufferBoundsChangeListener initWithParent:]):
(-[WebRootSampleBufferBoundsChangeListener dealloc]):
(-[WebRootSampleBufferBoundsChangeListener invalidate]):
(-[WebRootSampleBufferBoundsChangeListener start]):
(-[WebRootSampleBufferBoundsChangeListener stop]):
(-[WebRootSampleBufferBoundsChangeListener observeValueForKeyPath:ofObject:change:context:]):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::MediaPlayerPrivateMediaStreamAVFObjC):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayers):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::rootLayer const):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayLayer):
(WebCore::MediaPlayerPrivateMediaStreamAVFObjC::rootLayerBoundsDidChange):
* platform/graphics/cg/PDFDocumentImage.cpp:
Unified build fix.

Source/WebKit:

Implement WebProcess SampleBufferDisplayLayer and GPUProcess RemoteSampleBufferDisplayLayer.
RemoteSampleBufferDisplayLayer is a wrapper around LocalSampleBufferDisplayLayer.

A synchronous IPC is used at creation time of the layer to make sure we have a shared layer synchronously.
Future refactoring should allow to remove that constraint as done in regular video playing.

Override SampleBufferDisplayLayer create function pointer when rendering is to be done in GPUProcess.

* DerivedSources-input.xcfilelist:
* DerivedSources-output.xcfilelist:
* DerivedSources.make:
* GPUProcess/GPUConnectionToWebProcess.cpp:
(WebKit::GPUConnectionToWebProcess::sampleBufferDisplayLayerManager):
(WebKit::GPUConnectionToWebProcess::didReceiveMessage):
(WebKit::GPUConnectionToWebProcess::didReceiveSyncMessage):
* GPUProcess/GPUConnectionToWebProcess.h:
* GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp: Added.
(WebKit::RemoteSampleBufferDisplayLayer::create):
(WebKit::RemoteSampleBufferDisplayLayer::RemoteSampleBufferDisplayLayer):
(WebKit::RemoteSampleBufferDisplayLayer::~RemoteSampleBufferDisplayLayer):
(WebKit::RemoteSampleBufferDisplayLayer::contextID):
(WebKit::RemoteSampleBufferDisplayLayer::bounds const):
(WebKit::RemoteSampleBufferDisplayLayer::updateDisplayMode):
(WebKit::RemoteSampleBufferDisplayLayer::updateAffineTransform):
(WebKit::RemoteSampleBufferDisplayLayer::updateBoundsAndPosition):
(WebKit::RemoteSampleBufferDisplayLayer::flush):
(WebKit::RemoteSampleBufferDisplayLayer::flushAndRemoveImage):
(WebKit::RemoteSampleBufferDisplayLayer::enqueueSample):
(WebKit::RemoteSampleBufferDisplayLayer::clearEnqueuedSamples):
(WebKit::RemoteSampleBufferDisplayLayer::messageSenderConnection const):
(WebKit::RemoteSampleBufferDisplayLayer::sampleBufferDisplayLayerStatusDidChange):
(WebKit::RemoteSampleBufferDisplayLayer::streamTime const):
* GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h: Added.
* GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in: Added.
* GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.cpp: Added.
(WebKit::RemoteSampleBufferDisplayLayerManager::RemoteSampleBufferDisplayLayerManager):
(WebKit::RemoteSampleBufferDisplayLayerManager::didReceiveLayerMessage):
(WebKit::RemoteSampleBufferDisplayLayerManager::createLayer):
(WebKit::RemoteSampleBufferDisplayLayerManager::releaseLayer):
* GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.h: Added.
* GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.messages.in: Added.
* Scripts/webkit/messages.py:
* Shared/WebCoreArgumentCoders.h:
* Shared/mac/WebCoreArgumentCodersMac.mm:
(IPC::ArgumentCoder<CGRect>::encode):
(IPC::ArgumentCoder<CGRect>::decode):
(IPC::ArgumentCoder<CGSize>::encode):
(IPC::ArgumentCoder<CGSize>::decode):
(IPC::ArgumentCoder<CGPoint>::encode):
(IPC::ArgumentCoder<CGPoint>::decode):
(IPC::ArgumentCoder<CGAffineTransform>::encode):
(IPC::ArgumentCoder<CGAffineTransform>::decode):
* Sources.txt:
* SourcesCocoa.txt:
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/GPU/GPUProcessConnection.cpp:
(WebKit::GPUProcessConnection::sampleBufferDisplayLayerManager):
(WebKit::GPUProcessConnection::didReceiveMessage):
* WebProcess/GPU/GPUProcessConnection.h:
* WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
(WebKit::RemoteMediaPlayerManager::updatePreferences):
* WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp: Added.
(WebKit::SampleBufferDisplayLayer::create):
(WebKit::SampleBufferDisplayLayer::SampleBufferDisplayLayer):
(WebKit::SampleBufferDisplayLayer::~SampleBufferDisplayLayer):
(WebKit::SampleBufferDisplayLayer::didFail const):
(WebKit::SampleBufferDisplayLayer::updateDisplayMode):
(WebKit::SampleBufferDisplayLayer::bounds const):
(WebKit::SampleBufferDisplayLayer::updateAffineTransform):
(WebKit::SampleBufferDisplayLayer::updateBoundsAndPosition):
(WebKit::SampleBufferDisplayLayer::flush):
(WebKit::SampleBufferDisplayLayer::flushAndRemoveImage):
(WebKit::SampleBufferDisplayLayer::enqueueSample):
(WebKit::SampleBufferDisplayLayer::clearEnqueuedSamples):
(WebKit::SampleBufferDisplayLayer::rootLayer):
(WebKit::SampleBufferDisplayLayer::setDidFail):
(WebKit::SampleBufferDisplayLayer::boundsChanged):
* WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h: Added.
* WebProcess/GPU/webrtc/SampleBufferDisplayLayer.messages.in: Added.
* WebProcess/GPU/webrtc/SampleBufferDisplayLayerIdentifier.h: Added.
* WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.cpp: Added.
(WebKit::SampleBufferDisplayLayerManager::didReceiveLayerMessage):
(WebKit::SampleBufferDisplayLayerManager::createLayer):
(WebKit::SampleBufferDisplayLayerManager::addLayer):
(WebKit::SampleBufferDisplayLayerManager::removeLayer):
* WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.h: Added.
* WebProcess/WebProcess.h:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254688 268f45cc-cd09-0410-ab3c-d52691b4dbfc

39 files changed:
Source/WebCore/ChangeLog
Source/WebCore/SourcesCocoa.txt
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/platform/graphics/avfoundation/SampleBufferDisplayLayer.cpp [new file with mode: 0644]
Source/WebCore/platform/graphics/avfoundation/SampleBufferDisplayLayer.h
Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.h
Source/WebCore/platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.mm
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm
Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h
Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm
Source/WebCore/platform/graphics/cg/PDFDocumentImage.cpp
Source/WebKit/ChangeLog
Source/WebKit/DerivedSources-input.xcfilelist
Source/WebKit/DerivedSources-output.xcfilelist
Source/WebKit/DerivedSources.make
Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.cpp [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.h [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.messages.in [new file with mode: 0644]
Source/WebKit/Scripts/webkit/messages.py
Source/WebKit/Shared/WebCoreArgumentCoders.h
Source/WebKit/Shared/mac/WebCoreArgumentCodersMac.mm
Source/WebKit/Sources.txt
Source/WebKit/SourcesCocoa.txt
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/GPU/GPUProcessConnection.cpp
Source/WebKit/WebProcess/GPU/GPUProcessConnection.h
Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.cpp
Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.messages.in [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerIdentifier.h [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.cpp [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.h [new file with mode: 0644]

index 2b23aca..145d497 100644 (file)
@@ -1,3 +1,59 @@
+2020-01-16  youenn fablet  <youenn@apple.com>
+
+        Add support for MediaStream video track rendering in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=206286
+
+        Reviewed by Eric Carlson.
+
+        Add a way for WebKit layer to create its own specific SampleBufferDisplayLayer.
+        WebKit layer can set a function pointer that will be used to create SampleBufferDisplayLayer.
+        If none is provided, LocalSampleBufferDisplayLayer is used.
+
+        Update MediaPlayerPrivateMediaStreamAVFObjC to monitor itself its root layer bounds change.
+        Conversely, LocalSampleBufferDisplayLayer no longer monitors its root layer bounds change.
+        This removes the need for SampleBufferDisplayLayer::Client to be notified of bounds change.
+        We add an API to update the root layer bounds of a LocalSampleBufferDisplayLayer as well,
+        this API being used by GPUProcess to mirror changes done to WebProcess root layer.
+
+        WebRTC incoming video tracks do not have presentation timestamps so we mark them as kCMSampleAttachmentKey_DisplayImmediately.
+        This is currently lost when doing IPC from WebProcess to NetworkProcess. Add this information for every enqueued sample in LocalSampleBufferDisplayLayer.
+        We might want in the future to serialize all attachments of the sample.
+
+        Manually tested.
+
+        * SourcesCocoa.txt:
+        * WebCore.xcodeproj/project.pbxproj:
+        * platform/graphics/avfoundation/SampleBufferDisplayLayer.cpp: Added.
+        (WebCore::SampleBufferDisplayLayer::setCreator):
+        (WebCore::SampleBufferDisplayLayer::create):
+        * platform/graphics/avfoundation/SampleBufferDisplayLayer.h:
+        * platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.h:
+        * platform/graphics/avfoundation/objc/LocalSampleBufferDisplayLayer.mm:
+        (-[WebAVSampleBufferStatusChangeListener invalidate]):
+        (-[WebAVSampleBufferStatusChangeListener start]):
+        (-[WebAVSampleBufferStatusChangeListener stop]):
+        (-[WebAVSampleBufferStatusChangeListener observeValueForKeyPath:ofObject:change:context:]):
+        (WebCore::LocalSampleBufferDisplayLayer::LocalSampleBufferDisplayLayer):
+        (WebCore::LocalSampleBufferDisplayLayer::~LocalSampleBufferDisplayLayer):
+        (WebCore::LocalSampleBufferDisplayLayer::updateBoundsAndPosition):
+        (WebCore::LocalSampleBufferDisplayLayer::enqueueSample):
+        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.h:
+        * platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaStreamAVFObjC.mm:
+        (-[WebRootSampleBufferBoundsChangeListener initWithParent:]):
+        (-[WebRootSampleBufferBoundsChangeListener dealloc]):
+        (-[WebRootSampleBufferBoundsChangeListener invalidate]):
+        (-[WebRootSampleBufferBoundsChangeListener start]):
+        (-[WebRootSampleBufferBoundsChangeListener stop]):
+        (-[WebRootSampleBufferBoundsChangeListener observeValueForKeyPath:ofObject:change:context:]):
+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::MediaPlayerPrivateMediaStreamAVFObjC):
+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC):
+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::ensureLayers):
+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::rootLayer const):
+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayLayer):
+        (WebCore::MediaPlayerPrivateMediaStreamAVFObjC::rootLayerBoundsDidChange):
+        * platform/graphics/cg/PDFDocumentImage.cpp:
+        Unified build fix.
+
 2020-01-16  Víctor Manuel Jáquez Leal  <vjaquez@igalia.com>
 
         [Texmap] Remove unused private variable
index 8906f57..8e2aadd 100644 (file)
@@ -230,6 +230,7 @@ platform/graphics/angle/ExtensionsGLANGLE.cpp @no-unify
 platform/graphics/angle/GraphicsContextGLANGLE.cpp @no-unify
 platform/graphics/angle/TemporaryANGLESetting.cpp @no-unify
 
+platform/graphics/avfoundation/SampleBufferDisplayLayer.cpp
 platform/graphics/avfoundation/AVTrackPrivateAVFObjCImpl.mm @no-unify
 platform/graphics/avfoundation/AudioSourceProviderAVFObjC.mm @no-unify
 platform/graphics/avfoundation/CDMPrivateMediaSourceAVFObjC.mm @no-unify
index 9095d52..810f0da 100644 (file)
                3FFFF9A7159D9A550020BBD5 /* WebKitCSSViewportRule.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = WebKitCSSViewportRule.h; sourceTree = "<group>"; };
                3FFFF9AB159D9B060020BBD5 /* ViewportStyleResolver.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ViewportStyleResolver.cpp; sourceTree = "<group>"; };
                3FFFF9AC159D9B060020BBD5 /* ViewportStyleResolver.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ViewportStyleResolver.h; sourceTree = "<group>"; };
+               41024FC823CF254F00FDF98E /* SampleBufferDisplayLayer.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = SampleBufferDisplayLayer.cpp; sourceTree = "<group>"; };
                410626A72280A22A006D1B59 /* JSAudioNodeCustom.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSAudioNodeCustom.cpp; sourceTree = "<group>"; };
                4107908A1FC3E4F20061B27A /* ClientOrigin.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ClientOrigin.h; sourceTree = "<group>"; };
                410938282347799A009428BA /* JSAbortControllerCustom.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = JSAbortControllerCustom.cpp; sourceTree = "<group>"; };
                                076F0D0A12B8192700C26AA4 /* MediaPlayerPrivateAVFoundation.h */,
                                CDBEAEAB19D92B6C00BEBA88 /* MediaSelectionGroupAVFObjC.h */,
                                CDBEAEAA19D92B6C00BEBA88 /* MediaSelectionGroupAVFObjC.mm */,
+                               41024FC823CF254F00FDF98E /* SampleBufferDisplayLayer.cpp */,
                                414598BE23C8AAB8002B9CC8 /* SampleBufferDisplayLayer.h */,
                                CD336F6317FA0A4D00DDDCD0 /* VideoTrackPrivateAVF.h */,
                                0709D7901AE5557E004E42F8 /* WebMediaSessionManagerMac.cpp */,
diff --git a/Source/WebCore/platform/graphics/avfoundation/SampleBufferDisplayLayer.cpp b/Source/WebCore/platform/graphics/avfoundation/SampleBufferDisplayLayer.cpp
new file mode 100644 (file)
index 0000000..3d11c29
--- /dev/null
@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "SampleBufferDisplayLayer.h"
+
+#include "LocalSampleBufferDisplayLayer.h"
+
+namespace WebCore {
+
+SampleBufferDisplayLayer::LayerCreator SampleBufferDisplayLayer::m_layerCreator = nullptr;
+void SampleBufferDisplayLayer::setCreator(LayerCreator creator)
+{
+    m_layerCreator = creator;
+}
+
+std::unique_ptr<SampleBufferDisplayLayer> SampleBufferDisplayLayer::create(Client& client, bool hideRootLayer, IntSize size)
+{
+    if (m_layerCreator)
+        return m_layerCreator(client, hideRootLayer, size);
+
+#if ENABLE(MEDIA_STREAM) && USE(AVFOUNDATION)
+    return LocalSampleBufferDisplayLayer::create(client, hideRootLayer, size);
+#else
+    return nullptr;
+#endif
+}
+
+}
index 4ff57d7..401a727 100644 (file)
@@ -42,11 +42,13 @@ public:
     public:
         virtual ~Client() = default;
         virtual void sampleBufferDisplayLayerStatusDidChange(SampleBufferDisplayLayer&) = 0;
-        virtual void sampleBufferDisplayLayerBoundsDidChange(SampleBufferDisplayLayer&) = 0;
         virtual WTF::MediaTime streamTime() const = 0;
     };
 
-    explicit SampleBufferDisplayLayer(Client&);
+    WEBCORE_EXPORT static std::unique_ptr<SampleBufferDisplayLayer> create(Client&, bool hideRootLayer, IntSize);
+    using LayerCreator = std::unique_ptr<SampleBufferDisplayLayer> (*)(Client&, bool hideRootLayer, IntSize);
+    WEBCORE_EXPORT static void setCreator(LayerCreator);
+
     virtual ~SampleBufferDisplayLayer() = default;
 
     virtual bool didFail() const = 0;
@@ -66,7 +68,12 @@ public:
     virtual PlatformLayer* rootLayer() = 0;
 
 protected:
+    explicit SampleBufferDisplayLayer(Client&);
+
     WeakPtr<Client> m_client;
+
+private:
+    static LayerCreator m_layerCreator;
 };
 
 inline SampleBufferDisplayLayer::SampleBufferDisplayLayer(Client& client)
index 29cdea4..6faa409 100644 (file)
@@ -40,7 +40,7 @@ namespace WebCore {
 class WEBCORE_EXPORT LocalSampleBufferDisplayLayer final : public SampleBufferDisplayLayer, public CanMakeWeakPtr<LocalSampleBufferDisplayLayer> {
     WTF_MAKE_FAST_ALLOCATED;
 public:
-    static std::unique_ptr<SampleBufferDisplayLayer> create(Client&, bool hideRootLayer, IntSize);
+    static std::unique_ptr<LocalSampleBufferDisplayLayer> create(Client&, bool hideRootLayer, IntSize);
 
     LocalSampleBufferDisplayLayer(RetainPtr<AVSampleBufferDisplayLayer>&&, Client&, bool hideRootLayer, IntSize);
     ~LocalSampleBufferDisplayLayer();
@@ -54,7 +54,8 @@ public:
 
     PlatformLayer* rootLayer() final;
 
-private:
+    void updateRootLayerBoundsAndPosition(CGRect, CGPoint);
+
     bool didFail() const final;
 
     void updateDisplayMode(bool hideDisplayLayer, bool hideRootLayer) final;
@@ -69,8 +70,7 @@ private:
     void enqueueSample(MediaSample&) final;
     void clearEnqueuedSamples() final;
 
-    void ensureLayers();
-
+private:
     void removeOldSamplesFromPendingQueue();
     void addSampleToPendingQueue(MediaSample&);
     void requestNotificationWhenReadyForVideoData();
index f0c9cc7..cffdb97 100644 (file)
@@ -37,6 +37,7 @@
 #import <QuartzCore/CATransaction.h>
 
 #import <wtf/MainThread.h>
+#import <wtf/cf/TypeCastsCF.h>
 
 #import <pal/cocoa/AVFoundationSoftLink.h>
 
@@ -48,8 +49,8 @@ using namespace WebCore;
 
 - (id)initWithParent:(LocalSampleBufferDisplayLayer*)callback;
 - (void)invalidate;
-- (void)beginObservingLayers;
-- (void)stopObservingLayers;
+- (void)start;
+- (void)stop;
 @end
 
 @implementation WebAVSampleBufferStatusChangeListener
@@ -72,38 +73,33 @@ using namespace WebCore;
 
 - (void)invalidate
 {
-    [self stopObservingLayers];
+    [self stop];
     _parent = nullptr;
 }
 
-- (void)beginObservingLayers
+- (void)start
 {
     ASSERT(_parent);
     ASSERT(_parent->displayLayer());
-    ASSERT(_parent->rootLayer());
 
     [_parent->displayLayer() addObserver:self forKeyPath:@"status" options:NSKeyValueObservingOptionNew context:nil];
     [_parent->displayLayer() addObserver:self forKeyPath:@"error" options:NSKeyValueObservingOptionNew context:nil];
-    [_parent->rootLayer() addObserver:self forKeyPath:@"bounds" options:NSKeyValueObservingOptionNew context:nil];
 }
 
-- (void)stopObservingLayers
+- (void)stop
 {
-    if (!_parent)
+    if (!_parent || !_parent->displayLayer())
         return;
 
-    if (_parent->displayLayer()) {
-        [_parent->displayLayer() removeObserver:self forKeyPath:@"status"];
-        [_parent->displayLayer() removeObserver:self forKeyPath:@"error"];
-    }
-    if (_parent->rootLayer())
-        [_parent->rootLayer() removeObserver:self forKeyPath:@"bounds"];
+    [_parent->displayLayer() removeObserver:self forKeyPath:@"status"];
+    [_parent->displayLayer() removeObserver:self forKeyPath:@"error"];
 }
 
 - (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
 {
     UNUSED_PARAM(context);
     UNUSED_PARAM(keyPath);
+    UNUSED_PARAM(change);
     ASSERT(_parent);
 
     if (!_parent)
@@ -132,29 +128,6 @@ using namespace WebCore;
             return;
         }
     }
-
-    if ([[change valueForKey:NSKeyValueChangeNotificationIsPriorKey] boolValue])
-        return;
-
-    if ((CALayer *)object == _parent->rootLayer()) {
-        if ([keyPath isEqualToString:@"bounds"]) {
-            if (!_parent)
-                return;
-
-            if (isMainThread()) {
-                _parent->rootLayerBoundsDidChange();
-                return;
-            }
-
-            callOnMainThread([protectedSelf = RetainPtr<WebAVSampleBufferStatusChangeListener>(self)] {
-                if (!protectedSelf->_parent)
-                    return;
-
-                protectedSelf->_parent->rootLayerBoundsDidChange();
-            });
-        }
-    }
-
 }
 @end
 
@@ -169,7 +142,7 @@ static void runWithoutAnimations(const WTF::Function<void()>& function)
     [CATransaction commit];
 }
 
-std::unique_ptr<SampleBufferDisplayLayer> LocalSampleBufferDisplayLayer::create(Client& client, bool hideRootLayer, IntSize size)
+std::unique_ptr<LocalSampleBufferDisplayLayer> LocalSampleBufferDisplayLayer::create(Client& client, bool hideRootLayer, IntSize size)
 {
     auto sampleBufferDisplayLayer = adoptNS([PAL::allocAVSampleBufferDisplayLayerInstance() init]);
     if (!sampleBufferDisplayLayer)
@@ -196,7 +169,7 @@ LocalSampleBufferDisplayLayer::LocalSampleBufferDisplayLayer(RetainPtr<AVSampleB
 
     m_rootLayer.get().bounds = CGRectMake(0, 0, size.width(), size.height());
 
-    [m_statusChangeListener beginObservingLayers];
+    [m_statusChangeListener start];
 
     [m_rootLayer addSublayer:m_sampleBufferDisplayLayer.get()];
 
@@ -208,7 +181,7 @@ LocalSampleBufferDisplayLayer::LocalSampleBufferDisplayLayer(RetainPtr<AVSampleB
 
 LocalSampleBufferDisplayLayer::~LocalSampleBufferDisplayLayer()
 {
-    [m_statusChangeListener stopObservingLayers];
+    m_statusChangeListener = nullptr;
 
     m_pendingVideoSampleQueue.clear();
 
@@ -235,14 +208,6 @@ void LocalSampleBufferDisplayLayer::layerErrorDidChange()
     // FIXME: Log error.
 }
 
-void LocalSampleBufferDisplayLayer::rootLayerBoundsDidChange()
-{
-    ASSERT(isMainThread());
-    if (!m_client)
-        return;
-    m_client->sampleBufferDisplayLayerBoundsDidChange(*this);
-}
-
 PlatformLayer* LocalSampleBufferDisplayLayer::displayLayer()
 {
     return m_sampleBufferDisplayLayer.get();
@@ -289,6 +254,16 @@ void LocalSampleBufferDisplayLayer::updateBoundsAndPosition(CGRect videoBounds,
     });
 }
 
+void LocalSampleBufferDisplayLayer::updateRootLayerBoundsAndPosition(CGRect videoBounds, CGPoint position)
+{
+    runWithoutAnimations([&] {
+        m_rootLayer.get().bounds = videoBounds;
+        m_rootLayer.get().position = position;
+        m_sampleBufferDisplayLayer.get().bounds = videoBounds;
+        m_sampleBufferDisplayLayer.get().position = position;
+    });
+}
+
 void LocalSampleBufferDisplayLayer::flush()
 {
     [m_sampleBufferDisplayLayer flush];
index d9af27a..5baec56 100644 (file)
@@ -36,7 +36,7 @@
 #include <wtf/LoggerHelper.h>
 
 OBJC_CLASS AVSampleBufferDisplayLayer;
-OBJC_CLASS WebAVSampleBufferStatusChangeListener;
+OBJC_CLASS WebRootSampleBufferBoundsChangeListener;
 
 namespace PAL {
 class Clock;
@@ -82,6 +82,9 @@ public:
     WTFLogChannel& logChannel() const final;
 #endif
 
+    PlatformLayer* rootLayer() const;
+    void rootLayerBoundsDidChange();
+
 private:
     // MediaPlayerPrivateInterface
 
@@ -244,13 +247,14 @@ private:
 
     // SampleBufferDisplayLayer::Client
     void sampleBufferDisplayLayerStatusDidChange(SampleBufferDisplayLayer&) final;
-    void sampleBufferDisplayLayerBoundsDidChange(SampleBufferDisplayLayer&) final;
 
 #if !RELEASE_LOG_DISABLED
     Ref<const Logger> m_logger;
     const void* m_logIdentifier;
 #endif
 
+    RetainPtr<WebRootSampleBufferBoundsChangeListener> m_boundsChangeListener;
+
     bool m_videoMirrored { false };
     bool m_playing { false };
     bool m_muted { false };
index df9e205..25f207a 100644 (file)
 #import <pal/cf/CoreMediaSoftLink.h>
 #import <pal/cocoa/AVFoundationSoftLink.h>
 
+@interface WebRootSampleBufferBoundsChangeListener : NSObject {
+    WebCore::MediaPlayerPrivateMediaStreamAVFObjC* _parent;
+}
+
+- (id)initWithParent:(WebCore::MediaPlayerPrivateMediaStreamAVFObjC*)callback;
+- (void)invalidate;
+- (void)start;
+- (void)stop;
+@end
+
+@implementation WebRootSampleBufferBoundsChangeListener
+
+- (id)initWithParent:(WebCore::MediaPlayerPrivateMediaStreamAVFObjC*)parent
+{
+    if (!(self = [super init]))
+        return nil;
+
+    _parent = parent;
+
+    return self;
+}
+
+- (void)dealloc
+{
+    [self invalidate];
+    [super dealloc];
+}
+
+- (void)invalidate
+{
+    [self stop];
+    _parent = nullptr;
+}
+
+- (void)start
+{
+    ASSERT(_parent);
+    ASSERT(_parent->rootLayer());
+
+    [_parent->rootLayer() addObserver:self forKeyPath:@"bounds" options:NSKeyValueObservingOptionNew context:nil];
+}
+
+- (void)stop
+{
+    if (!_parent)
+        return;
+
+    if (_parent->rootLayer())
+        [_parent->rootLayer() removeObserver:self forKeyPath:@"bounds"];
+}
+
+- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
+{
+    UNUSED_PARAM(context);
+    UNUSED_PARAM(keyPath);
+    ASSERT(_parent);
+
+    if (!_parent)
+        return;
+
+    if ([[change valueForKey:NSKeyValueChangeNotificationIsPriorKey] boolValue])
+        return;
+
+    if ((CALayer *)object == _parent->rootLayer()) {
+        if ([keyPath isEqualToString:@"bounds"]) {
+            if (!_parent)
+                return;
+
+            if (isMainThread()) {
+                _parent->rootLayerBoundsDidChange();
+                return;
+            }
+
+            callOnMainThread([protectedSelf = RetainPtr<WebRootSampleBufferBoundsChangeListener>(self)] {
+                if (!protectedSelf->_parent)
+                    return;
+
+                protectedSelf->_parent->rootLayerBoundsDidChange();
+            });
+        }
+    }
+
+}
+@end
+
 namespace WebCore {
 using namespace PAL;
 
@@ -64,6 +149,7 @@ MediaPlayerPrivateMediaStreamAVFObjC::MediaPlayerPrivateMediaStreamAVFObjC(Media
     , m_logger(player->mediaPlayerLogger())
     , m_logIdentifier(player->mediaPlayerLogIdentifier())
 #endif
+    , m_boundsChangeListener(adoptNS([[WebRootSampleBufferBoundsChangeListener alloc] initWithParent:this]))
 {
     INFO_LOG(LOGIDENTIFIER);
 }
@@ -82,6 +168,8 @@ MediaPlayerPrivateMediaStreamAVFObjC::~MediaPlayerPrivateMediaStreamAVFObjC()
             track->removeObserver(*this);
     }
 
+    [m_boundsChangeListener stop];
+
     destroyLayers();
 
     auto audioTrackMap = WTFMove(m_audioTrackMap);
@@ -284,7 +372,7 @@ void MediaPlayerPrivateMediaStreamAVFObjC::ensureLayers()
         return;
 
     auto size = snappedIntRect(m_player->playerContentBoxRect()).size();
-    m_sampleBufferDisplayLayer = LocalSampleBufferDisplayLayer::create(*this, hideRootLayer(), size);
+    m_sampleBufferDisplayLayer = SampleBufferDisplayLayer::create(*this, hideRootLayer(), size);
 
     if (!m_sampleBufferDisplayLayer) {
         ERROR_LOG(LOGIDENTIFIER, "Creating the SampleBufferDisplayLayer failed.");
@@ -295,6 +383,8 @@ void MediaPlayerPrivateMediaStreamAVFObjC::ensureLayers()
     updateDisplayLayer();
 
     m_videoFullscreenLayerManager->setVideoLayer(m_sampleBufferDisplayLayer->rootLayer(), size);
+
+    [m_boundsChangeListener start];
 }
 
 void MediaPlayerPrivateMediaStreamAVFObjC::destroyLayers()
@@ -365,6 +455,11 @@ void MediaPlayerPrivateMediaStreamAVFObjC::prepareToPlay()
     INFO_LOG(LOGIDENTIFIER);
 }
 
+PlatformLayer* MediaPlayerPrivateMediaStreamAVFObjC::rootLayer() const
+{
+    return m_sampleBufferDisplayLayer ? m_sampleBufferDisplayLayer->rootLayer() : nullptr;
+}
+
 PlatformLayer* MediaPlayerPrivateMediaStreamAVFObjC::platformLayer() const
 {
     if (!m_sampleBufferDisplayLayer || !m_sampleBufferDisplayLayer->rootLayer() || m_displayMode == None)
@@ -948,7 +1043,7 @@ void MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayLayer()
     if (!m_sampleBufferDisplayLayer)
         return;
 
-    auto bounds = m_sampleBufferDisplayLayer->bounds();
+    auto bounds = rootLayer().bounds;
     auto videoBounds = bounds;
     if (m_videoRotation == MediaSample::VideoRotation::Right || m_videoRotation == MediaSample::VideoRotation::Left)
         std::swap(videoBounds.size.width, videoBounds.size.height);
@@ -956,7 +1051,7 @@ void MediaPlayerPrivateMediaStreamAVFObjC::updateDisplayLayer()
     m_sampleBufferDisplayLayer->updateBoundsAndPosition(videoBounds, { bounds.size.width / 2, bounds.size.height / 2});
 }
 
-void MediaPlayerPrivateMediaStreamAVFObjC::sampleBufferDisplayLayerBoundsDidChange(SampleBufferDisplayLayer&)
+void MediaPlayerPrivateMediaStreamAVFObjC::rootLayerBoundsDidChange()
 {
     updateDisplayLayer();
 }
index 4b97a9a..f855e19 100644 (file)
@@ -39,6 +39,8 @@ public:
     static Ref<MediaSampleAVFObjC> create(CMSampleBufferRef sample, VideoRotation rotation = VideoRotation::None, bool mirrored = false) { return adoptRef(*new MediaSampleAVFObjC(sample, rotation, mirrored)); }
     static RefPtr<MediaSampleAVFObjC> createImageSample(Vector<uint8_t>&&, unsigned long width, unsigned long height);
 
+    WEBCORE_EXPORT static void setAsDisplayImmediately(MediaSample&);
+
     RefPtr<JSC::Uint8ClampedArray> getRGBAImageData() const override;
 
     MediaTime presentationTime() const override;
index 17151a5..e894b91 100644 (file)
@@ -311,6 +311,15 @@ RefPtr<JSC::Uint8ClampedArray> MediaSampleAVFObjC::getRGBAImageData() const
 #endif
 }
 
+void MediaSampleAVFObjC::setAsDisplayImmediately(MediaSample& sample)
+{
+    CFArrayRef attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sample.platformSample().sample.cmSampleBuffer, true);
+    for (CFIndex i = 0; i < CFArrayGetCount(attachmentsArray); ++i) {
+        CFMutableDictionaryRef attachments = checked_cf_cast<CFMutableDictionaryRef>(CFArrayGetValueAtIndex(attachmentsArray, i));
+        CFDictionarySetValue(attachments, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);
+    }
+}
+
 String MediaSampleAVFObjC::toJSONString() const
 {
     auto object = JSON::Object::create();
index 80f6b18..2821ef8 100644 (file)
@@ -42,6 +42,7 @@
 #include "SharedBuffer.h"
 #include <CoreGraphics/CGContext.h>
 #include <CoreGraphics/CGPDFDocument.h>
+#include <pal/spi/cg/CoreGraphicsSPI.h>
 #include <wtf/MathExtras.h>
 #include <wtf/RAMSize.h>
 #include <wtf/RetainPtr.h>
index be6367f..1f7b182 100644 (file)
@@ -1,3 +1,98 @@
+2020-01-16  youenn fablet  <youenn@apple.com>
+
+        Add support for MediaStream video track rendering in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=206286
+
+        Reviewed by Eric Carlson.
+
+        Implement WebProcess SampleBufferDisplayLayer and GPUProcess RemoteSampleBufferDisplayLayer.
+        RemoteSampleBufferDisplayLayer is a wrapper around LocalSampleBufferDisplayLayer.
+
+        A synchronous IPC is used at creation time of the layer to make sure we have a shared layer synchronously.
+        Future refactoring should allow to remove that constraint as done in regular video playing.
+
+        Override SampleBufferDisplayLayer create function pointer when rendering is to be done in GPUProcess.
+
+        * DerivedSources-input.xcfilelist:
+        * DerivedSources-output.xcfilelist:
+        * DerivedSources.make:
+        * GPUProcess/GPUConnectionToWebProcess.cpp:
+        (WebKit::GPUConnectionToWebProcess::sampleBufferDisplayLayerManager):
+        (WebKit::GPUConnectionToWebProcess::didReceiveMessage):
+        (WebKit::GPUConnectionToWebProcess::didReceiveSyncMessage):
+        * GPUProcess/GPUConnectionToWebProcess.h:
+        * GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp: Added.
+        (WebKit::RemoteSampleBufferDisplayLayer::create):
+        (WebKit::RemoteSampleBufferDisplayLayer::RemoteSampleBufferDisplayLayer):
+        (WebKit::RemoteSampleBufferDisplayLayer::~RemoteSampleBufferDisplayLayer):
+        (WebKit::RemoteSampleBufferDisplayLayer::contextID):
+        (WebKit::RemoteSampleBufferDisplayLayer::bounds const):
+        (WebKit::RemoteSampleBufferDisplayLayer::updateDisplayMode):
+        (WebKit::RemoteSampleBufferDisplayLayer::updateAffineTransform):
+        (WebKit::RemoteSampleBufferDisplayLayer::updateBoundsAndPosition):
+        (WebKit::RemoteSampleBufferDisplayLayer::flush):
+        (WebKit::RemoteSampleBufferDisplayLayer::flushAndRemoveImage):
+        (WebKit::RemoteSampleBufferDisplayLayer::enqueueSample):
+        (WebKit::RemoteSampleBufferDisplayLayer::clearEnqueuedSamples):
+        (WebKit::RemoteSampleBufferDisplayLayer::messageSenderConnection const):
+        (WebKit::RemoteSampleBufferDisplayLayer::sampleBufferDisplayLayerStatusDidChange):
+        (WebKit::RemoteSampleBufferDisplayLayer::streamTime const):
+        * GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h: Added.
+        * GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in: Added.
+        * GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.cpp: Added.
+        (WebKit::RemoteSampleBufferDisplayLayerManager::RemoteSampleBufferDisplayLayerManager):
+        (WebKit::RemoteSampleBufferDisplayLayerManager::didReceiveLayerMessage):
+        (WebKit::RemoteSampleBufferDisplayLayerManager::createLayer):
+        (WebKit::RemoteSampleBufferDisplayLayerManager::releaseLayer):
+        * GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.h: Added.
+        * GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.messages.in: Added.
+        * Scripts/webkit/messages.py:
+        * Shared/WebCoreArgumentCoders.h:
+        * Shared/mac/WebCoreArgumentCodersMac.mm:
+        (IPC::ArgumentCoder<CGRect>::encode):
+        (IPC::ArgumentCoder<CGRect>::decode):
+        (IPC::ArgumentCoder<CGSize>::encode):
+        (IPC::ArgumentCoder<CGSize>::decode):
+        (IPC::ArgumentCoder<CGPoint>::encode):
+        (IPC::ArgumentCoder<CGPoint>::decode):
+        (IPC::ArgumentCoder<CGAffineTransform>::encode):
+        (IPC::ArgumentCoder<CGAffineTransform>::decode):
+        * Sources.txt:
+        * SourcesCocoa.txt:
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/GPU/GPUProcessConnection.cpp:
+        (WebKit::GPUProcessConnection::sampleBufferDisplayLayerManager):
+        (WebKit::GPUProcessConnection::didReceiveMessage):
+        * WebProcess/GPU/GPUProcessConnection.h:
+        * WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
+        (WebKit::RemoteMediaPlayerManager::updatePreferences):
+        * WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp: Added.
+        (WebKit::SampleBufferDisplayLayer::create):
+        (WebKit::SampleBufferDisplayLayer::SampleBufferDisplayLayer):
+        (WebKit::SampleBufferDisplayLayer::~SampleBufferDisplayLayer):
+        (WebKit::SampleBufferDisplayLayer::didFail const):
+        (WebKit::SampleBufferDisplayLayer::updateDisplayMode):
+        (WebKit::SampleBufferDisplayLayer::bounds const):
+        (WebKit::SampleBufferDisplayLayer::updateAffineTransform):
+        (WebKit::SampleBufferDisplayLayer::updateBoundsAndPosition):
+        (WebKit::SampleBufferDisplayLayer::flush):
+        (WebKit::SampleBufferDisplayLayer::flushAndRemoveImage):
+        (WebKit::SampleBufferDisplayLayer::enqueueSample):
+        (WebKit::SampleBufferDisplayLayer::clearEnqueuedSamples):
+        (WebKit::SampleBufferDisplayLayer::rootLayer):
+        (WebKit::SampleBufferDisplayLayer::setDidFail):
+        (WebKit::SampleBufferDisplayLayer::boundsChanged):
+        * WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h: Added.
+        * WebProcess/GPU/webrtc/SampleBufferDisplayLayer.messages.in: Added.
+        * WebProcess/GPU/webrtc/SampleBufferDisplayLayerIdentifier.h: Added.
+        * WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.cpp: Added.
+        (WebKit::SampleBufferDisplayLayerManager::didReceiveLayerMessage):
+        (WebKit::SampleBufferDisplayLayerManager::createLayer):
+        (WebKit::SampleBufferDisplayLayerManager::addLayer):
+        (WebKit::SampleBufferDisplayLayerManager::removeLayer):
+        * WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.h: Added.
+        * WebProcess/WebProcess.h:
+
 2020-01-16  David Kilzer  <ddkilzer@apple.com>
 
         Enable -Wconditional-uninitialized in WebInspectorUI, WebKitLegacy, WebKit projects
index 1a24bbf..98081aa 100644 (file)
@@ -24,6 +24,8 @@ $(PROJECT_DIR)/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in
 $(PROJECT_DIR)/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in
 $(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
 $(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in
+$(PROJECT_DIR)/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in
+$(PROJECT_DIR)/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.messages.in
 $(PROJECT_DIR)/NetworkProcess/Cookies/WebCookieManager.messages.in
 $(PROJECT_DIR)/NetworkProcess/CustomProtocols/LegacyCustomProtocolManager.messages.in
 $(PROJECT_DIR)/NetworkProcess/IndexedDB/WebIDBServer.messages.in
@@ -108,6 +110,7 @@ $(PROJECT_DIR)/WebProcess/GPU/GPUProcessConnection.messages.in
 $(PROJECT_DIR)/WebProcess/GPU/media/RemoteMediaPlayerManager.messages.in
 $(PROJECT_DIR)/WebProcess/GPU/webrtc/LibWebRTCCodecs.messages.in
 $(PROJECT_DIR)/WebProcess/GPU/webrtc/LibWebRTCRemoteCodecs.messages.in
+$(PROJECT_DIR)/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.messages.in
 $(PROJECT_DIR)/WebProcess/Geolocation/WebGeolocationManager.messages.in
 $(PROJECT_DIR)/WebProcess/Network/NetworkProcessConnection.messages.in
 $(PROJECT_DIR)/WebProcess/Network/WebResourceLoader.messages.in
index 1dc5465..9b60d15 100644 (file)
@@ -151,6 +151,12 @@ $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaResourceManagerMessagesR
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteObjectRegistryMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteObjectRegistryMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteObjectRegistryMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteSampleBufferDisplayLayerManagerMessageReceiver.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteSampleBufferDisplayLayerManagerMessages.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteSampleBufferDisplayLayerManagerMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteSampleBufferDisplayLayerMessageReceiver.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteSampleBufferDisplayLayerMessages.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteSampleBufferDisplayLayerMessagesReplies.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteScrollingCoordinatorMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteScrollingCoordinatorMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteScrollingCoordinatorMessagesReplies.h
@@ -160,6 +166,9 @@ $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteWebInspectorProxyMessagesRepl
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteWebInspectorUIMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteWebInspectorUIMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteWebInspectorUIMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/SampleBufferDisplayLayerMessageReceiver.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/SampleBufferDisplayLayerMessages.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/SampleBufferDisplayLayerMessagesReplies.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/SecItemShimProxyMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/SecItemShimProxyMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/SecItemShimProxyMessagesReplies.h
index a19d079..c34af5c 100644 (file)
@@ -150,9 +150,12 @@ MESSAGE_RECEIVERS = \
     RemoteMediaRecorderManager \
     RemoteMediaResourceManager \
     RemoteObjectRegistry \
+    RemoteSampleBufferDisplayLayer \
+    RemoteSampleBufferDisplayLayerManager \
     RemoteScrollingCoordinator \
     RemoteWebInspectorProxy \
     RemoteWebInspectorUI \
+    SampleBufferDisplayLayer \
     SecItemShimProxy \
     ServiceWorkerFetchTask \
     SmartMagnificationController \
index 46b1349..c0948d0 100644 (file)
@@ -49,6 +49,9 @@
 #include "RemoteMediaRecorderMessages.h"
 #include "RemoteMediaResourceManager.h"
 #include "RemoteMediaResourceManagerMessages.h"
+#include "RemoteSampleBufferDisplayLayerManager.h"
+#include "RemoteSampleBufferDisplayLayerManagerMessages.h"
+#include "RemoteSampleBufferDisplayLayerMessages.h"
 #include "RemoteScrollingCoordinatorTransaction.h"
 #include "UserMediaCaptureManagerProxy.h"
 #include "UserMediaCaptureManagerProxyMessages.h"
@@ -162,6 +165,14 @@ RemoteAudioMediaStreamTrackRendererManager& GPUConnectionToWebProcess::audioTrac
 
     return *m_audioTrackRendererManager;
 }
+
+RemoteSampleBufferDisplayLayerManager& GPUConnectionToWebProcess::sampleBufferDisplayLayerManager()
+{
+    if (!m_sampleBufferDisplayLayerManager)
+        m_sampleBufferDisplayLayerManager = makeUnique<RemoteSampleBufferDisplayLayerManager>(m_connection.copyRef());
+
+    return *m_sampleBufferDisplayLayerManager;
+}
 #endif
 #endif
 
@@ -209,6 +220,14 @@ void GPUConnectionToWebProcess::didReceiveMessage(IPC::Connection& connection, I
         audioTrackRendererManager().didReceiveRendererMessage(connection, decoder);
         return;
     }
+    if (decoder.messageReceiverName() == Messages::RemoteSampleBufferDisplayLayerManager::messageReceiverName()) {
+        sampleBufferDisplayLayerManager().didReceiveMessageFromWebProcess(connection, decoder);
+        return;
+    }
+    if (decoder.messageReceiverName() == Messages::RemoteSampleBufferDisplayLayer::messageReceiverName()) {
+        sampleBufferDisplayLayerManager().didReceiveLayerMessage(connection, decoder);
+        return;
+    }
 #endif
 #endif
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
@@ -231,6 +250,12 @@ void GPUConnectionToWebProcess::didReceiveSyncMessage(IPC::Connection& connectio
         userMediaCaptureManagerProxy().didReceiveSyncMessageFromGPUProcess(connection, decoder, replyEncoder);
         return;
     }
+#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK)
+    if (decoder.messageReceiverName() == Messages::RemoteSampleBufferDisplayLayerManager::messageReceiverName()) {
+        sampleBufferDisplayLayerManager().didReceiveSyncMessageFromWebProcess(connection, decoder, replyEncoder);
+        return;
+    }
+#endif
 #endif
 
     ASSERT_NOT_REACHED();
index cc2110b..b98d62f 100644 (file)
@@ -44,6 +44,7 @@ class RemoteAudioMediaStreamTrackRendererManager;
 class RemoteMediaPlayerManagerProxy;
 class RemoteMediaRecorderManager;
 class RemoteMediaResourceManager;
+class RemoteSampleBufferDisplayLayerManager;
 class UserMediaCaptureManagerProxy;
 
 class GPUConnectionToWebProcess
@@ -76,6 +77,7 @@ private:
     RemoteMediaRecorderManager& mediaRecorderManager();
 #if ENABLE(VIDEO_TRACK)
     RemoteAudioMediaStreamTrackRendererManager& audioTrackRendererManager();
+    RemoteSampleBufferDisplayLayerManager& sampleBufferDisplayLayerManager();
 #endif
 #endif
 
@@ -98,6 +100,7 @@ private:
     std::unique_ptr<RemoteMediaRecorderManager> m_remoteMediaRecorderManager;
 #if ENABLE(VIDEO_TRACK)
     std::unique_ptr<RemoteAudioMediaStreamTrackRendererManager> m_audioTrackRendererManager;
+    std::unique_ptr<RemoteSampleBufferDisplayLayerManager> m_sampleBufferDisplayLayerManager;
 #endif
 #endif
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp
new file mode 100644 (file)
index 0000000..4456d6f
--- /dev/null
@@ -0,0 +1,143 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "RemoteSampleBufferDisplayLayer.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "SampleBufferDisplayLayerMessages.h"
+#include <WebCore/ImageTransferSessionVT.h>
+#include <WebCore/LocalSampleBufferDisplayLayer.h>
+#include <WebCore/MediaSampleAVFObjC.h>
+#include <WebCore/RemoteVideoSample.h>
+
+using namespace WebCore;
+
+namespace WebKit {
+
+std::unique_ptr<RemoteSampleBufferDisplayLayer> RemoteSampleBufferDisplayLayer::create(SampleBufferDisplayLayerIdentifier identifier, Ref<IPC::Connection>&& connection, bool hideRootLayer, IntSize size)
+{
+    auto layer = std::unique_ptr<RemoteSampleBufferDisplayLayer>(new RemoteSampleBufferDisplayLayer(identifier, WTFMove(connection), hideRootLayer, size));
+    return layer->m_sampleBufferDisplayLayer ? WTFMove(layer) : nullptr;
+}
+
+RemoteSampleBufferDisplayLayer::RemoteSampleBufferDisplayLayer(SampleBufferDisplayLayerIdentifier identifier, Ref<IPC::Connection>&& connection, bool hideRootLayer, IntSize size)
+    : m_identifier(identifier)
+    , m_connection(WTFMove(connection))
+    , m_sampleBufferDisplayLayer(LocalSampleBufferDisplayLayer::create(*this, hideRootLayer, size))
+{
+    ASSERT(m_sampleBufferDisplayLayer);
+    if (!m_sampleBufferDisplayLayer)
+        return;
+
+    m_layerHostingContext = LayerHostingContext::createForExternalHostingProcess();
+    m_layerHostingContext->setRootLayer(m_sampleBufferDisplayLayer->rootLayer());
+}
+
+RemoteSampleBufferDisplayLayer::~RemoteSampleBufferDisplayLayer()
+{
+}
+
+Optional<LayerHostingContextID> RemoteSampleBufferDisplayLayer::contextID()
+{
+    return m_layerHostingContext->contextID();
+}
+
+CGRect RemoteSampleBufferDisplayLayer::bounds() const
+{
+    return m_sampleBufferDisplayLayer->bounds();
+}
+
+void RemoteSampleBufferDisplayLayer::updateDisplayMode(bool hideDisplayLayer, bool hideRootLayer)
+{
+    m_sampleBufferDisplayLayer->updateDisplayMode(hideDisplayLayer, hideRootLayer);
+}
+
+void RemoteSampleBufferDisplayLayer::updateAffineTransform(CGAffineTransform transform)
+{
+    m_sampleBufferDisplayLayer->updateAffineTransform(transform);
+}
+
+void RemoteSampleBufferDisplayLayer::updateBoundsAndPosition(CGRect bounds, CGPoint position)
+{
+    m_sampleBufferDisplayLayer->updateRootLayerBoundsAndPosition(bounds, position);
+}
+
+void RemoteSampleBufferDisplayLayer::flush()
+{
+    m_sampleBufferDisplayLayer->flush();
+}
+
+void RemoteSampleBufferDisplayLayer::flushAndRemoveImage()
+{
+    m_sampleBufferDisplayLayer->flushAndRemoveImage();
+}
+
+void RemoteSampleBufferDisplayLayer::enqueueSample(WebCore::RemoteVideoSample&& remoteSample)
+{
+    m_mediaTime = remoteSample.time();
+
+    if (!m_imageTransferSession || m_imageTransferSession->pixelFormat() != remoteSample.videoFormat())
+        m_imageTransferSession = ImageTransferSessionVT::create(remoteSample.videoFormat());
+
+    ASSERT(m_imageTransferSession);
+    if (!m_imageTransferSession)
+        return;
+
+    auto sample = m_imageTransferSession->createMediaSample(remoteSample.surface(), remoteSample.time(), remoteSample.size());
+
+    ASSERT(sample);
+    if (!sample)
+        return;
+
+    MediaSampleAVFObjC::setAsDisplayImmediately(*sample);
+    m_sampleBufferDisplayLayer->enqueueSample(*sample);
+}
+
+void RemoteSampleBufferDisplayLayer::clearEnqueuedSamples()
+{
+    m_sampleBufferDisplayLayer->clearEnqueuedSamples();
+}
+
+IPC::Connection* RemoteSampleBufferDisplayLayer::messageSenderConnection() const
+{
+    return m_connection.ptr();
+}
+
+void RemoteSampleBufferDisplayLayer::sampleBufferDisplayLayerStatusDidChange(WebCore::SampleBufferDisplayLayer&)
+{
+    send(Messages::SampleBufferDisplayLayer::SetDidFail { m_sampleBufferDisplayLayer->didFail() });
+}
+
+WTF::MediaTime RemoteSampleBufferDisplayLayer::streamTime() const
+{
+    // This is only an approximation which will clear all samples enqueued in m_sampleBufferDisplayLayer except the one being pushed.
+    return m_mediaTime;
+}
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.h
new file mode 100644 (file)
index 0000000..685a669
--- /dev/null
@@ -0,0 +1,85 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "LayerHostingContext.h"
+#include "MessageReceiver.h"
+#include "MessageSender.h"
+#include "SampleBufferDisplayLayerIdentifier.h"
+#include <WebCore/SampleBufferDisplayLayer.h>
+
+namespace WebCore {
+class ImageTransferSessionVT;
+class LocalSampleBufferDisplayLayer;
+class RemoteVideoSample;
+};
+
+namespace WebKit {
+
+class RemoteSampleBufferDisplayLayer : public WebCore::SampleBufferDisplayLayer::Client, public IPC::MessageReceiver, private IPC::MessageSender {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    static std::unique_ptr<RemoteSampleBufferDisplayLayer> create(SampleBufferDisplayLayerIdentifier, Ref<IPC::Connection>&&, bool hideRootLayer, WebCore::IntSize);
+    ~RemoteSampleBufferDisplayLayer();
+
+    // IPC::MessageReceiver
+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
+
+    Optional<LayerHostingContextID> contextID();
+    CGRect bounds() const;
+    
+private:
+    RemoteSampleBufferDisplayLayer(SampleBufferDisplayLayerIdentifier, Ref<IPC::Connection>&&, bool hideRootLayer, WebCore::IntSize);
+
+    void updateDisplayMode(bool hideDisplayLayer, bool hideRootLayer);
+    void updateAffineTransform(CGAffineTransform);
+    void updateBoundsAndPosition(CGRect, CGPoint);
+    void flush();
+    void flushAndRemoveImage();
+    void enqueueSample(WebCore::RemoteVideoSample&&);
+    void clearEnqueuedSamples();
+
+    // IPC::MessageSender
+    IPC::Connection* messageSenderConnection() const final;
+    uint64_t messageSenderDestinationID() const final { return m_identifier.toUInt64(); }
+
+    // WebCore::SampleBufferDisplayLayer::Client
+    void sampleBufferDisplayLayerStatusDidChange(WebCore::SampleBufferDisplayLayer&) final;
+    WTF::MediaTime streamTime() const final;
+
+    SampleBufferDisplayLayerIdentifier m_identifier;
+    Ref<IPC::Connection> m_connection;
+    std::unique_ptr<WebCore::ImageTransferSessionVT> m_imageTransferSession;
+    std::unique_ptr<WebCore::LocalSampleBufferDisplayLayer> m_sampleBufferDisplayLayer;
+    std::unique_ptr<LayerHostingContext> m_layerHostingContext;
+    WTF::MediaTime m_mediaTime;
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.messages.in
new file mode 100644 (file)
index 0000000..b7bbeff
--- /dev/null
@@ -0,0 +1,36 @@
+# Copyright (C) 2020 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+messages -> RemoteSampleBufferDisplayLayer NotRefCounted {
+    UpdateDisplayMode(bool hideDisplayLayer, bool hideRootLayer)
+    UpdateAffineTransform(CGAffineTransform transform)
+    UpdateBoundsAndPosition(CGRect bounds, CGPoint position)
+    Flush()
+    FlushAndRemoveImage()
+    EnqueueSample(WebCore::RemoteVideoSample sample)
+    ClearEnqueuedSamples()
+}
+
+#endif
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.cpp
new file mode 100644 (file)
index 0000000..4b9be49
--- /dev/null
@@ -0,0 +1,69 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "RemoteSampleBufferDisplayLayerManager.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "Decoder.h"
+#include "RemoteSampleBufferDisplayLayer.h"
+
+namespace WebKit {
+
+RemoteSampleBufferDisplayLayerManager::RemoteSampleBufferDisplayLayerManager(Ref<IPC::Connection>&& connection)
+    : m_connection(WTFMove(connection))
+{
+}
+
+RemoteSampleBufferDisplayLayerManager::~RemoteSampleBufferDisplayLayerManager() = default;
+
+void RemoteSampleBufferDisplayLayerManager::didReceiveLayerMessage(IPC::Connection& connection, IPC::Decoder& decoder)
+{
+    if (auto* layer = m_layers.get(makeObjectIdentifier<SampleBufferDisplayLayerIdentifierType>(decoder.destinationID())))
+        layer->didReceiveMessage(connection, decoder);
+}
+
+// FIXME: We should refactor code to use an asynchronous IPC.
+void RemoteSampleBufferDisplayLayerManager::createLayer(SampleBufferDisplayLayerIdentifier identifier, bool hideRootLayer, IntSize size, Messages::RemoteSampleBufferDisplayLayerManager::CreateLayerDelayedReply&& reply)
+{
+    ASSERT(!m_layers.contains(identifier));
+    auto layer = RemoteSampleBufferDisplayLayer::create(identifier, m_connection.copyRef(), hideRootLayer, size);
+    if (!layer)
+        return reply({ }, { });
+
+    reply(layer->contextID(), layer->bounds());
+    m_layers.add(identifier, WTFMove(layer));
+}
+
+void RemoteSampleBufferDisplayLayerManager::releaseLayer(SampleBufferDisplayLayerIdentifier identifier)
+{
+    ASSERT(m_layers.contains(identifier));
+    m_layers.remove(identifier);
+}
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.h b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.h
new file mode 100644 (file)
index 0000000..9f9b30c
--- /dev/null
@@ -0,0 +1,68 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "MessageReceiver.h"
+#include "RemoteSampleBufferDisplayLayerManagerMessagesReplies.h"
+#include "SampleBufferDisplayLayerIdentifier.h"
+#include <wtf/HashMap.h>
+
+namespace IPC {
+class Connection;
+class Decoder;
+}
+
+namespace WebKit {
+
+class RemoteSampleBufferDisplayLayer;
+
+class RemoteSampleBufferDisplayLayerManager final : private IPC::MessageReceiver {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    explicit RemoteSampleBufferDisplayLayerManager(Ref<IPC::Connection>&&);
+    ~RemoteSampleBufferDisplayLayerManager();
+
+    void didReceiveLayerMessage(IPC::Connection&, IPC::Decoder&);
+    void didReceiveMessageFromWebProcess(IPC::Connection& connection, IPC::Decoder& decoder) { didReceiveMessage(connection, decoder); }
+    void didReceiveSyncMessageFromWebProcess(IPC::Connection& connection, IPC::Decoder& decoder, std::unique_ptr<IPC::Encoder>& encoder) { didReceiveSyncMessage(connection, decoder, encoder); }
+
+private:
+    // IPC::MessageReceiver
+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
+    void didReceiveSyncMessage(IPC::Connection&, IPC::Decoder&, std::unique_ptr<IPC::Encoder>&) final;
+
+    void createLayer(SampleBufferDisplayLayerIdentifier, bool hideRootLayer, WebCore::IntSize, Messages::RemoteSampleBufferDisplayLayerManager::CreateLayerDelayedReply&&);
+    void releaseLayer(SampleBufferDisplayLayerIdentifier);
+
+    Ref<IPC::Connection> m_connection;
+    HashMap<SampleBufferDisplayLayerIdentifier, std::unique_ptr<RemoteSampleBufferDisplayLayer>> m_layers;
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.messages.in b/Source/WebKit/GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.messages.in
new file mode 100644 (file)
index 0000000..0449533
--- /dev/null
@@ -0,0 +1,31 @@
+# Copyright (C) 2020 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+messages -> RemoteSampleBufferDisplayLayerManager NotRefCounted {
+    CreateLayer(WebKit::SampleBufferDisplayLayerIdentifier id, bool hideRootLayer, WebCore::IntSize size) -> (Optional<WebKit::LayerHostingContextID> contextID, CGRect bounds) Synchronous
+    ReleaseLayer(WebKit::SampleBufferDisplayLayerIdentifier id)
+}
+
+#endif
index 2564f52..dc9e14f 100644 (file)
@@ -225,6 +225,7 @@ def types_that_cannot_be_forward_declared():
         'WebKit::RemoteMediaResourceIdentifier',
         'WebKit::RTCDecoderIdentifier',
         'WebKit::RTCEncoderIdentifier',
+        'WebKit::SampleBufferDisplayLayerIdentifier',
         'WebKit::StorageAreaIdentifier',
         'WebKit::StorageAreaImplIdentifier',
         'WebKit::StorageNamespaceIdentifier',
index 84d439c..7bf71c1 100644 (file)
@@ -446,6 +446,26 @@ template<> struct ArgumentCoder<WebCore::KeypressCommand> {
     static void encode(Encoder&, const WebCore::KeypressCommand&);
     static Optional<WebCore::KeypressCommand> decode(Decoder&);
 };
+
+template<> struct ArgumentCoder<CGPoint> {
+    static void encode(Encoder&, CGPoint);
+    static Optional<CGPoint> decode(Decoder&);
+};
+
+template<> struct ArgumentCoder<CGSize> {
+    static void encode(Encoder&, CGSize);
+    static Optional<CGSize> decode(Decoder&);
+};
+
+template<> struct ArgumentCoder<CGRect> {
+    static void encode(Encoder&, CGRect);
+    static Optional<CGRect> decode(Decoder&);
+};
+
+template<> struct ArgumentCoder<CGAffineTransform> {
+    static void encode(Encoder&, CGAffineTransform);
+    static Optional<CGAffineTransform> decode(Decoder&);
+};
 #endif
 
 #if PLATFORM(IOS_FAMILY)
index 5ba52c9..8cbdafb 100644 (file)
@@ -548,6 +548,79 @@ Optional<WebCore::KeypressCommand> ArgumentCoder<WebCore::KeypressCommand>::deco
     return WTFMove(command);
 }
 
+void ArgumentCoder<CGRect>::encode(Encoder& encoder, CGRect rect)
+{
+    encoder << rect.origin << rect.size;
+}
+
+Optional<CGRect> ArgumentCoder<CGRect>::decode(Decoder& decoder)
+{
+    Optional<CGPoint> origin;
+    decoder >> origin;
+    if (!origin)
+        return { };
+
+    Optional<CGSize> size;
+    decoder >> size;
+    if (!size)
+        return { };
+
+    return CGRect { *origin, *size };
+}
+
+void ArgumentCoder<CGSize>::encode(Encoder& encoder, CGSize size)
+{
+    encoder << size.width << size.height;
+}
+
+Optional<CGSize> ArgumentCoder<CGSize>::decode(Decoder& decoder)
+{
+    CGSize size;
+    if (!decoder.decode(size.width))
+        return { };
+    if (!decoder.decode(size.height))
+        return { };
+    return size;
+}
+
+void ArgumentCoder<CGPoint>::encode(Encoder& encoder, CGPoint point)
+{
+    encoder << point.x << point.y;
+}
+
+Optional<CGPoint> ArgumentCoder<CGPoint>::decode(Decoder& decoder)
+{
+    CGPoint point;
+    if (!decoder.decode(point.x))
+        return { };
+    if (!decoder.decode(point.y))
+        return { };
+    return point;
+}
+
+void ArgumentCoder<CGAffineTransform>::encode(Encoder& encoder, CGAffineTransform transform)
+{
+    encoder << transform.a << transform.b << transform.c << transform.d << transform.tx << transform.ty;
+}
+
+Optional<CGAffineTransform> ArgumentCoder<CGAffineTransform>::decode(Decoder& decoder)
+{
+    CGAffineTransform transform;
+    if (!decoder.decode(transform.a))
+        return { };
+    if (!decoder.decode(transform.b))
+        return { };
+    if (!decoder.decode(transform.c))
+        return { };
+    if (!decoder.decode(transform.d))
+        return { };
+    if (!decoder.decode(transform.tx))
+        return { };
+    if (!decoder.decode(transform.ty))
+        return { };
+    return transform;
+}
+
 #if ENABLE(CONTENT_FILTERING)
 
 void ArgumentCoder<WebCore::ContentFilterUnblockHandler>::encode(Encoder& encoder, const WebCore::ContentFilterUnblockHandler& contentFilterUnblockHandler)
index 979fa65..3da2800 100644 (file)
@@ -28,6 +28,8 @@ GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp
 GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp
 GPUProcess/webrtc/RemoteMediaRecorder.cpp
 GPUProcess/webrtc/RemoteMediaRecorderManager.cpp
+GPUProcess/webrtc/RemoteSampleBufferDisplayLayer.cpp
+GPUProcess/webrtc/RemoteSampleBufferDisplayLayerManager.cpp
 GPUProcess/media/RemoteAudioTrackProxy.cpp
 GPUProcess/media/RemoteMediaPlayerManagerProxy.cpp
 GPUProcess/media/RemoteMediaPlayerProxy.cpp
@@ -529,6 +531,8 @@ WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
 WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
 WebProcess/GPU/webrtc/MediaRecorderProvider.cpp
 WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp
+WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp
+WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.cpp
 
 WebProcess/Network/NetworkProcessConnection.cpp
 WebProcess/Network/WebLoaderStrategy.cpp
index b23832a..6a8e2e6 100644 (file)
@@ -644,5 +644,8 @@ RemoteAudioMediaStreamTrackRendererManagerMessageReceiver.cpp
 RemoteAudioMediaStreamTrackRendererMessageReceiver.cpp
 RemoteMediaRecorderMessageReceiver.cpp
 RemoteMediaRecorderManagerMessageReceiver.cpp
+RemoteSampleBufferDisplayLayerManagerMessageReceiver.cpp
+RemoteSampleBufferDisplayLayerMessageReceiver.cpp
+SampleBufferDisplayLayerMessageReceiver.cpp
 ServiceWorkerFetchTaskMessageReceiver.cpp
 TextCheckingControllerProxyMessageReceiver.cpp
index 5c06117..11fcdc6 100644 (file)
                3F915C0E1F564DED00183CE9 /* WKFullScreenWindowControllerIOS.mm */ = {isa = PBXFileReference; indentWidth = 4; lastKnownFileType = sourcecode.cpp.objcpp; name = WKFullScreenWindowControllerIOS.mm; path = ios/fullscreen/WKFullScreenWindowControllerIOS.mm; sourceTree = "<group>"; };
                3F915C0F1F564DED00183CE9 /* WKFullScreenWindowControllerIOS.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = WKFullScreenWindowControllerIOS.h; path = ios/fullscreen/WKFullScreenWindowControllerIOS.h; sourceTree = "<group>"; };
                3FB08E421F60B240005E5312 /* iOS.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = iOS.xcassets; sourceTree = "<group>"; };
+               41024FC023CF104500FDF98E /* SampleBufferDisplayLayerManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SampleBufferDisplayLayerManager.h; sourceTree = "<group>"; };
+               41024FC123CF104500FDF98E /* SampleBufferDisplayLayerManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SampleBufferDisplayLayerManager.cpp; sourceTree = "<group>"; };
                4103FBA12061BDB700C2EAF8 /* NetworkLoadChecker.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkLoadChecker.h; sourceTree = "<group>"; };
                4103FBA22061BDB800C2EAF8 /* NetworkLoadChecker.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkLoadChecker.cpp; sourceTree = "<group>"; };
                410482CB1DDD2FB500F006D0 /* RTCNetwork.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RTCNetwork.cpp; sourceTree = "<group>"; };
                41FCD6AD23CCC26000C62567 /* AudioMediaStreamTrackRenderer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioMediaStreamTrackRenderer.h; sourceTree = "<group>"; };
                41FCD6AE23CCC26100C62567 /* AudioMediaStreamTrackRenderer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AudioMediaStreamTrackRenderer.cpp; sourceTree = "<group>"; };
                41FCD6AF23CCC26100C62567 /* AudioMediaStreamTrackRendererIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioMediaStreamTrackRendererIdentifier.h; sourceTree = "<group>"; };
+               41FCD6B323CDC60400C62567 /* SampleBufferDisplayLayer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SampleBufferDisplayLayer.h; sourceTree = "<group>"; };
+               41FCD6B423CDCAC200C62567 /* RemoteSampleBufferDisplayLayer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteSampleBufferDisplayLayer.h; sourceTree = "<group>"; };
+               41FCD6B523CDE1C600C62567 /* RemoteSampleBufferDisplayLayer.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteSampleBufferDisplayLayer.cpp; sourceTree = "<group>"; };
+               41FCD6B623CDE1C600C62567 /* RemoteSampleBufferDisplayLayer.messages.in */ = {isa = PBXFileReference; lastKnownFileType = text; path = RemoteSampleBufferDisplayLayer.messages.in; sourceTree = "<group>"; };
+               41FCD6B723CDE53D00C62567 /* SampleBufferDisplayLayer.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = SampleBufferDisplayLayer.cpp; sourceTree = "<group>"; };
+               41FCD6B823CDE53E00C62567 /* SampleBufferDisplayLayer.messages.in */ = {isa = PBXFileReference; lastKnownFileType = text; path = SampleBufferDisplayLayer.messages.in; sourceTree = "<group>"; };
+               41FCD6BC23CE031800C62567 /* SampleBufferDisplayLayerIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SampleBufferDisplayLayerIdentifier.h; sourceTree = "<group>"; };
+               41FCD6BD23CE043F00C62567 /* RemoteSampleBufferDisplayLayerManager.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RemoteSampleBufferDisplayLayerManager.messages.in; sourceTree = "<group>"; };
+               41FCD6BE23CE044000C62567 /* RemoteSampleBufferDisplayLayerManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteSampleBufferDisplayLayerManager.cpp; sourceTree = "<group>"; };
+               41FCD6BF23CE044000C62567 /* RemoteSampleBufferDisplayLayerManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteSampleBufferDisplayLayerManager.h; sourceTree = "<group>"; };
                4450AEBF1DC3FAE5009943F2 /* SharedMemoryCocoa.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SharedMemoryCocoa.cpp; sourceTree = "<group>"; };
                4459984122833E6000E61373 /* SyntheticEditingCommandType.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = SyntheticEditingCommandType.h; sourceTree = "<group>"; };
                449D90D821FD63FE00F677C0 /* LocalAuthenticationSoftLink.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = LocalAuthenticationSoftLink.mm; sourceTree = "<group>"; };
                                4176E89223C34D6E003E83FE /* MediaRecorderProvider.h */,
                                4172198C23B612E800AE5686 /* RTCDecoderIdentifier.h */,
                                4158649A23BE092400A0A61E /* RTCEncoderIdentifier.h */,
+                               41FCD6B723CDE53D00C62567 /* SampleBufferDisplayLayer.cpp */,
+                               41FCD6B323CDC60400C62567 /* SampleBufferDisplayLayer.h */,
+                               41FCD6B823CDE53E00C62567 /* SampleBufferDisplayLayer.messages.in */,
+                               41FCD6BC23CE031800C62567 /* SampleBufferDisplayLayerIdentifier.h */,
+                               41024FC123CF104500FDF98E /* SampleBufferDisplayLayerManager.cpp */,
+                               41024FC023CF104500FDF98E /* SampleBufferDisplayLayerManager.h */,
                        );
                        path = webrtc;
                        sourceTree = "<group>";
                                4176E89A23C36F71003E83FE /* RemoteMediaRecorderManager.cpp */,
                                4176E89923C36D8E003E83FE /* RemoteMediaRecorderManager.h */,
                                4176E89B23C37222003E83FE /* RemoteMediaRecorderManager.messages.in */,
+                               41FCD6B523CDE1C600C62567 /* RemoteSampleBufferDisplayLayer.cpp */,
+                               41FCD6B423CDCAC200C62567 /* RemoteSampleBufferDisplayLayer.h */,
+                               41FCD6B623CDE1C600C62567 /* RemoteSampleBufferDisplayLayer.messages.in */,
+                               41FCD6BE23CE044000C62567 /* RemoteSampleBufferDisplayLayerManager.cpp */,
+                               41FCD6BF23CE044000C62567 /* RemoteSampleBufferDisplayLayerManager.h */,
+                               41FCD6BD23CE043F00C62567 /* RemoteSampleBufferDisplayLayerManager.messages.in */,
                        );
                        path = webrtc;
                        sourceTree = "<group>";
                                41DE7C6C22278F1E00532B65 /* ServiceWorkerFetchTask.cpp in Sources */,
                                2D92A787212B6AB100F493FD /* ShareableBitmap.cpp in Sources */,
                                A1ADAFB62368E6A8009CB776 /* SharedMemory.cpp in Sources */,
+                               575B1BB923CE9C0B0020639A /* SimulatedInputDispatcher.cpp in Sources */,
                                2DE6943D18BD2A68005C15E5 /* SmartMagnificationControllerMessageReceiver.cpp in Sources */,
                                1A334DED16DE8F88006A8E38 /* StorageAreaMapMessageReceiver.cpp in Sources */,
                                9368EEDE2303A90200BDB11A /* StorageManagerSetMessageReceiver.cpp in Sources */,
                                2D11B7D52126A283006F8878 /* UnifiedSource72.cpp in Sources */,
                                2D54C311212F3B330049C174 /* UnifiedSource72-mm.mm in Sources */,
                                2D54C310212F3B330049C174 /* UnifiedSource73-mm.mm in Sources */,
-                               575B1BBA23CE9C130020639A /* WebAutomationSession.cpp in Sources */,
                                2D11B7D62126A283006F8878 /* UnifiedSource73.cpp in Sources */,
                                2D54C315212F3B330049C174 /* UnifiedSource74-mm.mm in Sources */,
                                2D11B7D72126A283006F8878 /* UnifiedSource74.cpp in Sources */,
                                1A60224C18C16B9F00C3E8C9 /* VisitedLinkStoreMessageReceiver.cpp in Sources */,
                                1A8E7D3C18C15149005A702A /* VisitedLinkTableControllerMessageReceiver.cpp in Sources */,
                                57DCED702142EE680016B847 /* WebAuthenticatorCoordinatorProxyMessageReceiver.cpp in Sources */,
+                               575B1BBA23CE9C130020639A /* WebAutomationSession.cpp in Sources */,
                                1C0A19571C90068F00FE0EBB /* WebAutomationSessionMessageReceiver.cpp in Sources */,
+                               575B1BB823CE9BFC0020639A /* WebAutomationSessionProxy.cpp in Sources */,
                                1C0A19531C8FFDFB00FE0EBB /* WebAutomationSessionProxyMessageReceiver.cpp in Sources */,
                                1A1FEC1C1627B45700700F6D /* WebConnectionMessageReceiver.cpp in Sources */,
                                330934471315B9220097A7BC /* WebCookieManagerMessageReceiver.cpp in Sources */,
                                BC0E606112D6BA910012A72A /* WebGeolocationManagerMessageReceiver.cpp in Sources */,
                                BC0E618212D6CB1D0012A72A /* WebGeolocationManagerProxyMessageReceiver.cpp in Sources */,
                                510523751C73D38F007993CB /* WebIDBConnectionToServerMessageReceiver.cpp in Sources */,
-                               575B1BB923CE9C0B0020639A /* SimulatedInputDispatcher.cpp in Sources */,
                                933E835B23A1B75000DEF289 /* WebIDBServerMessageReceiver.cpp in Sources */,
                                FEDBDCD61E68D20000A59F8F /* WebInspectorInterruptDispatcherMessageReceiver.cpp in Sources */,
                                1C8E2A351277852400BC7BD0 /* WebInspectorMessageReceiver.cpp in Sources */,
                                637281A321ADC744009E0DE6 /* WKDownloadProgress.mm in Sources */,
                                5CE9120D2293C219005BEC78 /* WKMain.mm in Sources */,
                                4657D88A22664A2F005DE823 /* WKOrientationAccessAlert.mm in Sources */,
-                               575B1BB823CE9BFC0020639A /* WebAutomationSessionProxy.cpp in Sources */,
                                5CA26D83217AD1B800F97A35 /* WKSafeBrowsingWarning.mm in Sources */,
                                1DB01944211CF005009FB3E8 /* WKShareSheet.mm in Sources */,
                                7A78FF332241919B0096483E /* WKStorageAccessAlert.mm in Sources */,
index 18e936d..53a85e6 100644 (file)
@@ -34,6 +34,7 @@
 #include "LibWebRTCCodecsMessages.h"
 #include "RemoteMediaPlayerManager.h"
 #include "RemoteMediaPlayerManagerMessages.h"
+#include "SampleBufferDisplayLayerMessages.h"
 #include "UserMediaCaptureManager.h"
 #include "UserMediaCaptureManagerMessages.h"
 #include "WebCoreArgumentCoders.h"
@@ -64,6 +65,15 @@ void GPUProcessConnection::didReceiveInvalidMessage(IPC::Connection&, IPC::Strin
 {
 }
 
+#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+SampleBufferDisplayLayerManager& GPUProcessConnection::sampleBufferDisplayLayerManager()
+{
+    if (!m_sampleBufferDisplayLayerManager)
+        m_sampleBufferDisplayLayerManager = makeUnique<SampleBufferDisplayLayerManager>();
+    return *m_sampleBufferDisplayLayerManager;
+}
+#endif
+
 void GPUProcessConnection::didReceiveMessage(IPC::Connection& connection, IPC::Decoder& decoder)
 {
     if (decoder.messageReceiverName() == Messages::RemoteMediaPlayerManager::messageReceiverName()) {
@@ -76,7 +86,13 @@ void GPUProcessConnection::didReceiveMessage(IPC::Connection& connection, IPC::D
             captureManager->didReceiveMessageFromGPUProcess(connection, decoder);
         return;
     }
-#endif
+#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK)
+    if (decoder.messageReceiverName() == Messages::SampleBufferDisplayLayer::messageReceiverName()) {
+        sampleBufferDisplayLayerManager().didReceiveLayerMessage(connection, decoder);
+        return;
+    }
+#endif // PLATFORM(COCOA) && ENABLE(VIDEO_TRACK)
+#endif // ENABLE(MEDIA_STREAM)
 #if USE(LIBWEBRTC) && PLATFORM(COCOA)
     if (decoder.messageReceiverName() == Messages::LibWebRTCCodecs::messageReceiverName()) {
         WebProcess::singleton().libWebRTCCodecs().didReceiveMessage(connection, decoder);
index a8ad4fe..4600e53 100644 (file)
@@ -28,6 +28,7 @@
 #if ENABLE(GPU_PROCESS)
 
 #include "Connection.h"
+#include "SampleBufferDisplayLayerManager.h"
 #include <wtf/RefCounted.h>
 #include <wtf/text/WTFString.h>
 
@@ -51,6 +52,9 @@ public:
     void setAuditToken(Optional<audit_token_t> auditToken) { m_auditToken = auditToken; }
     Optional<audit_token_t> auditToken() const { return m_auditToken; }
 #endif
+#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+    SampleBufferDisplayLayerManager& sampleBufferDisplayLayerManager();
+#endif
 
 private:
     GPUProcessConnection(IPC::Connection::Identifier);
@@ -66,6 +70,9 @@ private:
 #if HAVE(AUDIT_TOKEN)
     Optional<audit_token_t> m_auditToken;
 #endif
+#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+    std::unique_ptr<SampleBufferDisplayLayerManager> m_sampleBufferDisplayLayerManager;
+#endif
 };
 
 } // namespace WebKit
index 547d45f..180ac6d 100644 (file)
@@ -370,8 +370,12 @@ void RemoteMediaPlayerManager::updatePreferences(const Settings& settings)
     RemoteMediaPlayerSupport::setRegisterRemotePlayerCallback(settings.useGPUProcessForMedia() ? WTFMove(registerEngine) : RemoteMediaPlayerSupport::RegisterRemotePlayerCallback());
 
 #if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
-    if (settings.useGPUProcessForMedia())
+    if (settings.useGPUProcessForMedia()) {
         WebCore::AudioMediaStreamTrackRenderer::setCreator(WebKit::AudioMediaStreamTrackRenderer::create);
+        WebCore::SampleBufferDisplayLayer::setCreator([](auto& client, bool hideRootLayer, auto size) {
+            return WebProcess::singleton().ensureGPUProcessConnection().sampleBufferDisplayLayerManager().createLayer(client, hideRootLayer, size);
+        });
+    }
 #endif
 }
 
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.cpp
new file mode 100644 (file)
index 0000000..ae0161a
--- /dev/null
@@ -0,0 +1,132 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "SampleBufferDisplayLayer.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "LayerHostingContext.h"
+#include "RemoteSampleBufferDisplayLayerManagerMessages.h"
+#include "RemoteSampleBufferDisplayLayerManagerMessagesReplies.h"
+#include "RemoteSampleBufferDisplayLayerMessages.h"
+#include "SampleBufferDisplayLayerManager.h"
+#include "WebCoreArgumentCoders.h"
+
+using namespace WebCore;
+
+namespace WebKit {
+
+std::unique_ptr<SampleBufferDisplayLayer> SampleBufferDisplayLayer::create(SampleBufferDisplayLayerManager& manager, Client& client, bool hideRootLayer, IntSize size)
+{
+    auto layer = std::unique_ptr<SampleBufferDisplayLayer>(new SampleBufferDisplayLayer(manager, client, hideRootLayer, size));
+    return layer->rootLayer() ? WTFMove(layer) : nullptr;
+}
+
+SampleBufferDisplayLayer::SampleBufferDisplayLayer(SampleBufferDisplayLayerManager& manager, Client& client, bool hideRootLayer, IntSize size)
+    : WebCore::SampleBufferDisplayLayer(client)
+    , m_manager(makeWeakPtr(manager))
+    , m_connection(WebProcess::singleton().ensureGPUProcessConnection().connection())
+    , m_identifier(SampleBufferDisplayLayerIdentifier::generate())
+{
+    manager.addLayer(*this);
+
+    Optional<LayerHostingContextID> contextId;
+    if (!m_connection->sendSync(Messages::RemoteSampleBufferDisplayLayerManager::CreateLayer { m_identifier, hideRootLayer, size }, Messages::RemoteSampleBufferDisplayLayerManager::CreateLayer::Reply { contextId, m_bounds }, 0))
+        return;
+
+    if (!contextId)
+        return;
+
+    m_videoLayer = LayerHostingContext::createPlatformLayerForHostingContext(*contextId);
+}
+
+SampleBufferDisplayLayer::~SampleBufferDisplayLayer()
+{
+    m_connection->send(Messages::RemoteSampleBufferDisplayLayerManager::ReleaseLayer { m_identifier }, 0);
+    if (m_manager)
+        m_manager->removeLayer(*this);
+}
+
+bool SampleBufferDisplayLayer::didFail() const
+{
+    return m_didFail;
+}
+
+void SampleBufferDisplayLayer::updateDisplayMode(bool hideDisplayLayer, bool hideRootLayer)
+{
+    m_connection->send(Messages::RemoteSampleBufferDisplayLayer::UpdateDisplayMode { hideDisplayLayer, hideRootLayer }, m_identifier);
+}
+
+CGRect SampleBufferDisplayLayer::bounds() const
+{
+    return m_bounds;
+}
+
+void SampleBufferDisplayLayer::updateAffineTransform(CGAffineTransform transform)
+{
+    m_connection->send(Messages::RemoteSampleBufferDisplayLayer::UpdateAffineTransform { transform }, m_identifier);
+}
+
+void SampleBufferDisplayLayer::updateBoundsAndPosition(CGRect bounds, CGPoint position)
+{
+    m_bounds = bounds;
+    m_connection->send(Messages::RemoteSampleBufferDisplayLayer::UpdateBoundsAndPosition { bounds, position }, m_identifier);
+}
+
+void SampleBufferDisplayLayer::flush()
+{
+    m_connection->send(Messages::RemoteSampleBufferDisplayLayer::Flush { }, m_identifier);
+}
+
+void SampleBufferDisplayLayer::flushAndRemoveImage()
+{
+    m_connection->send(Messages::RemoteSampleBufferDisplayLayer::FlushAndRemoveImage { }, m_identifier);
+}
+
+void SampleBufferDisplayLayer::enqueueSample(MediaSample& sample)
+{
+    if (auto remoteSample = RemoteVideoSample::create(sample))
+        m_connection->send(Messages::RemoteSampleBufferDisplayLayer::EnqueueSample { *remoteSample }, m_identifier);
+}
+
+void SampleBufferDisplayLayer::clearEnqueuedSamples()
+{
+    m_connection->send(Messages::RemoteSampleBufferDisplayLayer::ClearEnqueuedSamples { }, m_identifier);
+}
+
+PlatformLayer* SampleBufferDisplayLayer::rootLayer()
+{
+    return m_videoLayer.get();
+}
+
+void SampleBufferDisplayLayer::setDidFail(bool value)
+{
+    m_didFail = value;
+}
+
+}
+
+#endif
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.h
new file mode 100644 (file)
index 0000000..8259c52
--- /dev/null
@@ -0,0 +1,76 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "MessageReceiver.h"
+#include "SampleBufferDisplayLayerIdentifier.h"
+#include <WebCore/SampleBufferDisplayLayer.h>
+#include <wtf/WeakPtr.h>
+
+namespace WebKit {
+
+class SampleBufferDisplayLayerManager;
+
+class SampleBufferDisplayLayer final : public WebCore::SampleBufferDisplayLayer, public IPC::MessageReceiver, public CanMakeWeakPtr<SampleBufferDisplayLayer> {
+public:
+    static std::unique_ptr<SampleBufferDisplayLayer> create(SampleBufferDisplayLayerManager&, Client&, bool hideRootLayer, WebCore::IntSize);
+    ~SampleBufferDisplayLayer();
+
+    SampleBufferDisplayLayerIdentifier identifier() const { return m_identifier; }
+
+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
+
+private:
+    SampleBufferDisplayLayer(SampleBufferDisplayLayerManager&, Client&, bool hideRootLayer, WebCore::IntSize);
+
+    // WebCore::SampleBufferDisplayLayer
+    bool didFail() const final;
+    void updateDisplayMode(bool hideDisplayLayer, bool hideRootLayer) final;
+    CGRect bounds() const final;
+    void updateAffineTransform(CGAffineTransform) final;
+    void updateBoundsAndPosition(CGRect, CGPoint) final;
+    void flush() final;
+    void flushAndRemoveImage() final;
+    void enqueueSample(WebCore::MediaSample&) final;
+    void clearEnqueuedSamples() final;
+    PlatformLayer* rootLayer() final;
+    
+    void setDidFail(bool);
+
+    WeakPtr<SampleBufferDisplayLayerManager> m_manager;
+    Ref<IPC::Connection> m_connection;
+    SampleBufferDisplayLayerIdentifier m_identifier;
+
+    RetainPtr<PlatformLayer> m_videoLayer;
+    CGRect m_bounds;
+    bool m_didFail { false };
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.messages.in b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayer.messages.in
new file mode 100644 (file)
index 0000000..6eeffaf
--- /dev/null
@@ -0,0 +1,30 @@
+# Copyright (C) 2020 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+messages -> SampleBufferDisplayLayer NotRefCounted {
+    SetDidFail(bool value)
+}
+
+#endif
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerIdentifier.h b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerIdentifier.h
new file mode 100644 (file)
index 0000000..98695ff
--- /dev/null
@@ -0,0 +1,39 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice , this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(GPU_PROCESS)
+
+#include <wtf/ObjectIdentifier.h>
+
+namespace WebKit {
+
+enum SampleBufferDisplayLayerIdentifierType { };
+using SampleBufferDisplayLayerIdentifier = ObjectIdentifier<SampleBufferDisplayLayerIdentifierType>;
+
+} // namespace WebKit
+
+#endif
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.cpp b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.cpp
new file mode 100644 (file)
index 0000000..4822f77
--- /dev/null
@@ -0,0 +1,65 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "SampleBufferDisplayLayerManager.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+using namespace WebCore;
+
+namespace WebKit {
+
+void SampleBufferDisplayLayerManager::didReceiveLayerMessage(IPC::Connection& connection, IPC::Decoder& decoder)
+{
+    if (auto* layer = m_layers.get(makeObjectIdentifier<SampleBufferDisplayLayerIdentifierType>(decoder.destinationID())).get())
+        layer->didReceiveMessage(connection, decoder);
+}
+
+std::unique_ptr<WebCore::SampleBufferDisplayLayer> SampleBufferDisplayLayerManager::createLayer(SampleBufferDisplayLayer::Client& client, bool hideRootLayer, WebCore::IntSize size)
+{
+    auto layer = SampleBufferDisplayLayer::create(*this, client, hideRootLayer, size);
+    if (!layer)
+        return { };
+
+    m_layers.add(layer->identifier(), makeWeakPtr(*layer));
+    return layer;
+}
+
+void SampleBufferDisplayLayerManager::addLayer(SampleBufferDisplayLayer& layer)
+{
+    ASSERT(!m_layers.contains(layer.identifier()));
+    m_layers.add(layer.identifier(), makeWeakPtr(layer));
+}
+
+void SampleBufferDisplayLayerManager::removeLayer(SampleBufferDisplayLayer& layer)
+{
+    ASSERT(m_layers.contains(layer.identifier()));
+    m_layers.remove(layer.identifier());
+}
+
+}
+
+#endif
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.h b/Source/WebKit/WebProcess/GPU/webrtc/SampleBufferDisplayLayerManager.h
new file mode 100644 (file)
index 0000000..ecc3af7
--- /dev/null
@@ -0,0 +1,53 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "SampleBufferDisplayLayer.h"
+#include <wtf/HashMap.h>
+
+namespace WebKit {
+
+class SampleBufferDisplayLayerManager : public CanMakeWeakPtr<SampleBufferDisplayLayerManager> {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    SampleBufferDisplayLayerManager() = default;
+    ~SampleBufferDisplayLayerManager() = default;
+
+    void addLayer(SampleBufferDisplayLayer&);
+    void removeLayer(SampleBufferDisplayLayer&);
+
+    void didReceiveLayerMessage(IPC::Connection&, IPC::Decoder&);
+    std::unique_ptr<WebCore::SampleBufferDisplayLayer> createLayer(SampleBufferDisplayLayer::Client&, bool hideRootLayer, WebCore::IntSize);
+
+private:
+    HashMap<SampleBufferDisplayLayerIdentifier, WeakPtr<SampleBufferDisplayLayer>> m_layers;
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)