[Media in GPU process] Implement the remote video layer support
authorpeng.liu6@apple.com <peng.liu6@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 15 Jan 2020 02:51:21 +0000 (02:51 +0000)
committerpeng.liu6@apple.com <peng.liu6@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 15 Jan 2020 02:51:21 +0000 (02:51 +0000)
https://bugs.webkit.org/show_bug.cgi?id=206043

Reviewed by Youenn Fablet.

Source/WebCore:

No new tests, covered by existing tests.

This patch adds function LayerHostingContext::createPlatformLayerForHostingContext() which is
a wrapper of [CALayer _web_renderLayerWithContextID:] to create WebVideoHostingLayer in the Web process.

It also moves LayerHostingContext.[h|mm] to from folder "mac" to "cocoa" because LayerHostingContext
is not only used on the Mac version.
Some issues related to unified build are also fixed.

* platform/graphics/ColorSpace.h:
* platform/graphics/LayoutRect.h:
(WebCore::LayoutRect::encode const):
(WebCore::LayoutRect::decode):
* platform/graphics/cocoa/WebCoreCALayerExtras.mm:
(+[CALayer _web_renderLayerWithContextID:]):
* platform/graphics/cv/ImageTransferSessionVT.mm:

Source/WebKit:

RemoteMediaPlayerProxy in the GPU process creates a LayerHostingContext with createForExternalHostingProcess()
and share the context ID with the MediaPlayerPrivateRemote in the Web process.
Then the MediaPlayerPrivateRemote creates a hosting layer with the context ID for rendering.

This patch updates RemoteMediaPlayerProxy::PrepareForPlayback to provide the initial video content box and scale
for the VideoFullscreenLayerManagerObjC on the GPU process to create WebVideoContainerLayer.
It adds XPC message RemoteMediaPlayerManager::FirstVideoFrameAvailable to notify the renderer on the Web process that the video layer is ready.
RemoteMediaPlayerManagerProxy::SetSize message is removed to avoid the unnecessary XPC message
because MediaPlayerPrivateAVFoundationObjC::setSize() does nothing.

* GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::prepareForPlayback):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerContentBoxRect const):
(WebKit::RemoteMediaPlayerProxy::mediaPlayerContentsScale const):
(WebKit::RemoteMediaPlayerProxy::setSize): Deleted.
* GPUProcess/media/RemoteMediaPlayerProxy.h:
* GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* Platform/cocoa/LayerHostingContext.h: Renamed from Source/WebKit/Platform/mac/LayerHostingContext.h.
* Platform/cocoa/LayerHostingContext.mm: Renamed from Source/WebKit/Platform/mac/LayerHostingContext.mm.
(WebKit::LayerHostingContext::createPlatformLayerForHostingContext):
* Platform/ios/PaymentAuthorizationController.mm:
* PluginProcess/mac/PluginControllerProxyMac.mm:
* SourcesCocoa.txt:
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
(WebKit::MediaPlayerPrivateRemote::prepareForPlayback):
(WebKit::MediaPlayerPrivateRemote::MediaPlayerPrivateRemote::load):
(WebKit::MediaPlayerPrivateRemote::firstVideoFrameAvailable):
(WebKit::MediaPlayerPrivateRemote::platformLayer const):
(WebKit::MediaPlayerPrivateRemote::setSize): Deleted.
* WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
(WebKit::RemoteMediaPlayerManager::characteristicChanged):
(WebKit::RemoteMediaPlayerManager::sizeChanged):
(WebKit::RemoteMediaPlayerManager::addRemoteVideoTrack):
(WebKit::RemoteMediaPlayerManager::removeRemoteVideoTrack):
(WebKit::RemoteMediaPlayerManager::remoteVideoTrackConfigurationChanged):
(WebKit::RemoteMediaPlayerManager::firstVideoFrameAvailable):
* WebProcess/GPU/media/RemoteMediaPlayerManager.h:
* WebProcess/GPU/media/RemoteMediaPlayerManager.messages.in:
* WebProcess/Plugins/Netscape/NetscapePlugin.cpp:

LayoutTests:

* gpu-process/TestExpectations:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254555 268f45cc-cd09-0410-ab3c-d52691b4dbfc

23 files changed:
LayoutTests/ChangeLog
LayoutTests/gpu-process/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/platform/graphics/ColorSpace.h
Source/WebCore/platform/graphics/LayoutRect.h
Source/WebCore/platform/graphics/cocoa/WebCoreCALayerExtras.mm
Source/WebCore/platform/graphics/cv/ImageTransferSessionVT.mm
Source/WebKit/ChangeLog
Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h
Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
Source/WebKit/Platform/cocoa/LayerHostingContext.h [moved from Source/WebKit/Platform/mac/LayerHostingContext.h with 95% similarity]
Source/WebKit/Platform/cocoa/LayerHostingContext.mm [moved from Source/WebKit/Platform/mac/LayerHostingContext.mm with 95% similarity]
Source/WebKit/Platform/ios/PaymentAuthorizationController.mm
Source/WebKit/PluginProcess/mac/PluginControllerProxyMac.mm
Source/WebKit/SourcesCocoa.txt
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp
Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h
Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.cpp
Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.h
Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.messages.in
Source/WebKit/WebProcess/Plugins/Netscape/NetscapePlugin.cpp

index afc5f20..66f2207 100644 (file)
@@ -1,3 +1,12 @@
+2020-01-14  Peng Liu  <peng.liu6@apple.com>
+
+        [Media in GPU process] Implement the remote video layer support
+        https://bugs.webkit.org/show_bug.cgi?id=206043
+
+        Reviewed by Youenn Fablet.
+
+        * gpu-process/TestExpectations:
+
 2020-01-14  Ryosuke Niwa  <rniwa@webkit.org>
 
         REGRESSION: fast/shadow-dom/link-element-in-shadow-tree.html is flaky
index de1afb4..7de7780 100644 (file)
@@ -237,6 +237,9 @@ http/tests/cache/disk-cache/disk-cache-media-small.html [ Skip ]
 http/tests/contentextensions/text-track-blocked.html [ Skip ]
 http/tests/contentextensions/video-element-resource-type.html [ Skip ]
 http/tests/media [ Skip ]
+http/tests/media/video-load-twice.html [ Pass ]
+http/tests/media/video-play-progress.html [ Pass ]
+http/tests/media/video-preload.html [ Pass ]
 http/tests/navigation/page-cache-mediakeysession.html [ Skip ]
 http/tests/navigation/page-cache-mediastream.html [ Skip ]
 http/wpt/mediarecorder [ Skip ]
index b1048f2..b843b49 100644 (file)
@@ -1,3 +1,27 @@
+2020-01-14  Peng Liu  <peng.liu6@apple.com>
+
+        [Media in GPU process] Implement the remote video layer support
+        https://bugs.webkit.org/show_bug.cgi?id=206043
+
+        Reviewed by Youenn Fablet.
+
+        No new tests, covered by existing tests.
+
+        This patch adds function LayerHostingContext::createPlatformLayerForHostingContext() which is
+        a wrapper of [CALayer _web_renderLayerWithContextID:] to create WebVideoHostingLayer in the Web process.
+
+        It also moves LayerHostingContext.[h|mm] to from folder "mac" to "cocoa" because LayerHostingContext
+        is not only used on the Mac version.
+        Some issues related to unified build are also fixed.
+
+        * platform/graphics/ColorSpace.h:
+        * platform/graphics/LayoutRect.h:
+        (WebCore::LayoutRect::encode const):
+        (WebCore::LayoutRect::decode):
+        * platform/graphics/cocoa/WebCoreCALayerExtras.mm:
+        (+[CALayer _web_renderLayerWithContextID:]):
+        * platform/graphics/cv/ImageTransferSessionVT.mm:
+
 2020-01-14  Dean Jackson  <dino@apple.com>
 
         Build ANGLE as a dynamic library
index 0aa274a..6fcd78f 100644 (file)
@@ -32,6 +32,5 @@ enum class ColorSpace : uint8_t {
     LinearRGB,
     DisplayP3
 };
-    
-} // namespace WebCore
 
+} // namespace WebCore
index cc73b8d..8246991 100644 (file)
@@ -57,7 +57,29 @@ public:
     LayoutRect(const IntRect& rect) : m_location(rect.location()), m_size(rect.size()) { }
     
     WEBCORE_EXPORT explicit LayoutRect(const FloatRect&); // don't do this implicitly since it's lossy
-        
+
+    template<class Encoder>
+    void encode(Encoder& encoder) const
+    {
+        encoder << m_location << m_size;
+    }
+
+    template<class Decoder>
+    static Optional<LayoutRect> decode(Decoder& decoder)
+    {
+        Optional<LayoutPoint> layoutPoint;
+        decoder >> layoutPoint;
+        if (!layoutPoint)
+            return WTF::nullopt;
+
+        Optional<LayoutSize> layoutSize;
+        decoder >> layoutSize;
+        if (!layoutSize)
+            return WTF::nullopt;
+
+        return {{ *layoutPoint, *layoutSize }};
+    }
+
     LayoutPoint location() const { return m_location; }
     LayoutSize size() const { return m_size; }
 
index 23bb0de..1805c0a 100644 (file)
@@ -81,6 +81,9 @@
 + (CALayer *)_web_renderLayerWithContextID:(uint32_t)contextID
 {
     CALayerHost *layerHost = [CALayerHost layer];
+#ifndef NDEBUG
+    [layerHost setName:@"WebVideoHostingLayer"];
+#endif
     layerHost.contextId = contextID;
     return layerHost;
 }
index 15397ab..fa8ee0b 100644 (file)
 
 #if USE(VIDEOTOOLBOX)
 
+#import "GraphicsContextCG.h"
 #import "Logging.h"
 #import "MediaSampleAVFObjC.h"
 #import <CoreMedia/CMFormatDescription.h>
 #import <CoreMedia/CMSampleBuffer.h>
+
+#import "CoreVideoSoftLink.h"
 #import <pal/cf/CoreMediaSoftLink.h>
+#import "VideoToolboxSoftLink.h"
 
 #if HAVE(IOSURFACE) && !PLATFORM(MACCATALYST)
 #include <pal/spi/cocoa/IOSurfaceSPI.h>
 #endif
 
-#import "CoreVideoSoftLink.h"
-
 namespace WebCore {
 using namespace PAL;
 
index 134a938..3746175 100644 (file)
@@ -1,3 +1,54 @@
+2020-01-14  Peng Liu  <peng.liu6@apple.com>
+
+        [Media in GPU process] Implement the remote video layer support
+        https://bugs.webkit.org/show_bug.cgi?id=206043
+
+        Reviewed by Youenn Fablet.
+
+        RemoteMediaPlayerProxy in the GPU process creates a LayerHostingContext with createForExternalHostingProcess()
+        and share the context ID with the MediaPlayerPrivateRemote in the Web process.
+        Then the MediaPlayerPrivateRemote creates a hosting layer with the context ID for rendering.
+
+        This patch updates RemoteMediaPlayerProxy::PrepareForPlayback to provide the initial video content box and scale
+        for the VideoFullscreenLayerManagerObjC on the GPU process to create WebVideoContainerLayer.
+        It adds XPC message RemoteMediaPlayerManager::FirstVideoFrameAvailable to notify the renderer on the Web process that the video layer is ready.
+        RemoteMediaPlayerManagerProxy::SetSize message is removed to avoid the unnecessary XPC message
+        because MediaPlayerPrivateAVFoundationObjC::setSize() does nothing.
+
+        * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
+        (WebKit::RemoteMediaPlayerProxy::prepareForPlayback):
+        (WebKit::RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable):
+        (WebKit::RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged):
+        (WebKit::RemoteMediaPlayerProxy::mediaPlayerContentBoxRect const):
+        (WebKit::RemoteMediaPlayerProxy::mediaPlayerContentsScale const):
+        (WebKit::RemoteMediaPlayerProxy::setSize): Deleted.
+        * GPUProcess/media/RemoteMediaPlayerProxy.h:
+        * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
+        * Platform/cocoa/LayerHostingContext.h: Renamed from Source/WebKit/Platform/mac/LayerHostingContext.h.
+        * Platform/cocoa/LayerHostingContext.mm: Renamed from Source/WebKit/Platform/mac/LayerHostingContext.mm.
+        (WebKit::LayerHostingContext::createPlatformLayerForHostingContext):
+        * Platform/ios/PaymentAuthorizationController.mm:
+        * PluginProcess/mac/PluginControllerProxyMac.mm:
+        * SourcesCocoa.txt:
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/GPU/media/MediaPlayerPrivateRemote.cpp:
+        (WebKit::MediaPlayerPrivateRemote::prepareForPlayback):
+        (WebKit::MediaPlayerPrivateRemote::MediaPlayerPrivateRemote::load):
+        (WebKit::MediaPlayerPrivateRemote::firstVideoFrameAvailable):
+        (WebKit::MediaPlayerPrivateRemote::platformLayer const):
+        (WebKit::MediaPlayerPrivateRemote::setSize): Deleted.
+        * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
+        * WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
+        (WebKit::RemoteMediaPlayerManager::characteristicChanged):
+        (WebKit::RemoteMediaPlayerManager::sizeChanged):
+        (WebKit::RemoteMediaPlayerManager::addRemoteVideoTrack):
+        (WebKit::RemoteMediaPlayerManager::removeRemoteVideoTrack):
+        (WebKit::RemoteMediaPlayerManager::remoteVideoTrackConfigurationChanged):
+        (WebKit::RemoteMediaPlayerManager::firstVideoFrameAvailable):
+        * WebProcess/GPU/media/RemoteMediaPlayerManager.h:
+        * WebProcess/GPU/media/RemoteMediaPlayerManager.messages.in:
+        * WebProcess/Plugins/Netscape/NetscapePlugin.cpp:
+
 2020-01-10  Jiewen Tan  <jiewen_tan@apple.com>
 
         [WebAuthn] Implement SPI to tell UI clients to select assertion responses
index 1c24193..a8ee858 100644 (file)
@@ -40,6 +40,7 @@
 #include "RemoteMediaResourceManager.h"
 #include "RemoteVideoTrackProxy.h"
 #include "WebCoreArgumentCoders.h"
+#include <WebCore/LayoutRect.h>
 #include <WebCore/MediaPlayer.h>
 #include <WebCore/MediaPlayerPrivate.h>
 #include <WebCore/NotImplemented.h>
@@ -105,12 +106,16 @@ void RemoteMediaPlayerProxy::load(const URL& url, Optional<SandboxExtension::Han
     completionHandler(WTFMove(configuration));
 }
 
-void RemoteMediaPlayerProxy::prepareForPlayback(bool privateMode, WebCore::MediaPlayerEnums::Preload preload, bool preservesPitch, bool prepareForRendering)
+void RemoteMediaPlayerProxy::prepareForPlayback(bool privateMode, WebCore::MediaPlayerEnums::Preload preload, bool preservesPitch, bool prepareForRendering, LayoutRect layoutRect, float videoContentScale, CompletionHandler<void(Optional<LayerHostingContextID>&& contextId)>&& completionHandler)
 {
     m_player->setPrivateBrowsingMode(privateMode);
     m_player->setPreload(preload);
     m_player->setPreservesPitch(preservesPitch);
     m_player->prepareForRendering();
+    m_videoContentBoxRect = layoutRect;
+    m_videoContentScale = videoContentScale;
+    m_layerHostingContext = LayerHostingContext::createForExternalHostingProcess();
+    completionHandler(m_layerHostingContext->contextID());
 }
 
 void RemoteMediaPlayerProxy::cancelLoad()
@@ -179,11 +184,6 @@ void RemoteMediaPlayerProxy::prepareForRendering()
     m_player->prepareForRendering();
 }
 
-void RemoteMediaPlayerProxy::setSize(const WebCore::IntSize& size)
-{
-    m_player->setSize(size);
-}
-
 void RemoteMediaPlayerProxy::setVisible(bool visible)
 {
     m_player->setVisible(visible);
@@ -430,12 +430,13 @@ void RemoteMediaPlayerProxy::mediaPlayerEngineUpdated()
 
 void RemoteMediaPlayerProxy::mediaPlayerFirstVideoFrameAvailable()
 {
-    notImplemented();
+    m_layerHostingContext->setRootLayer(m_player->platformLayer());
+    m_webProcessConnection->send(Messages::RemoteMediaPlayerManager::FirstVideoFrameAvailable(m_id), 0);
 }
 
 void RemoteMediaPlayerProxy::mediaPlayerRenderingModeChanged()
 {
-    notImplemented();
+    m_layerHostingContext->setRootLayer(m_player->platformLayer());
 }
 
 void RemoteMediaPlayerProxy::mediaPlayerActiveSourceBuffersChanged()
@@ -496,14 +497,12 @@ bool RemoteMediaPlayerProxy::mediaPlayerIsFullscreenPermitted() const
 
 LayoutRect RemoteMediaPlayerProxy::mediaPlayerContentBoxRect() const
 {
-    notImplemented();
-    return LayoutRect();
+    return m_videoContentBoxRect;
 }
 
 float RemoteMediaPlayerProxy::mediaPlayerContentsScale() const
 {
-    notImplemented();
-    return 1;
+    return m_videoContentScale;
 }
 
 void RemoteMediaPlayerProxy::mediaPlayerPause()
index b5fccd7..5ba783a 100644 (file)
@@ -52,6 +52,8 @@ class VideoTrackPrivate;
 
 namespace WebKit {
 
+using LayerHostingContextID = uint32_t;
+class LayerHostingContext;
 class RemoteAudioTrackProxy;
 class RemoteMediaPlayerManagerProxy;
 class RemoteVideoTrackProxy;
@@ -72,7 +74,8 @@ public:
 
     void getConfiguration(RemoteMediaPlayerConfiguration&);
 
-    void prepareForPlayback(bool privateMode, WebCore::MediaPlayerEnums::Preload, bool preservesPitch, bool prepareForRendering);
+    void prepareForPlayback(bool privateMode, WebCore::MediaPlayerEnums::Preload, bool preservesPitch, bool prepareForRendering, WebCore::LayoutRect, float videoContentScale, CompletionHandler<void(Optional<LayerHostingContextID>&&)>&&);
+    void prepareForRendering();
 
     void load(const URL&, Optional<SandboxExtension::Handle>&&, const WebCore::ContentType&, const String&, CompletionHandler<void(RemoteMediaPlayerConfiguration&&)>&&);
     void cancelLoad();
@@ -92,8 +95,6 @@ public:
     void setPrivateBrowsingMode(bool);
     void setPreservesPitch(bool);
 
-    void prepareForRendering();
-    void setSize(const WebCore::IntSize&);
     void setVisible(bool);
     void setShouldMaintainAspectRatio(bool);
     void setVideoFullscreenFrame(WebCore::FloatRect);
@@ -209,6 +210,7 @@ private:
     RefPtr<SandboxExtension> m_sandboxExtension;
     Ref<IPC::Connection> m_webProcessConnection;
     RefPtr<WebCore::MediaPlayer> m_player;
+    std::unique_ptr<LayerHostingContext> m_layerHostingContext;
     RemoteMediaPlayerManagerProxy& m_manager;
     WebCore::MediaPlayerEnums::MediaEngineIdentifier m_engineIdentifier;
     Vector<WebCore::ContentType> m_typesRequiringHardwareSupport;
@@ -218,6 +220,8 @@ private:
     bool m_seekableChanged { true };
     bool m_bufferedChanged { true };
     bool m_renderingCanBeAccelerated { true };
+    WebCore::LayoutRect m_videoContentBoxRect;
+    float m_videoContentScale { 1.0 };
 
 #if !RELEASE_LOG_DISABLED
     const Logger& m_logger;
index 162453b..cdb4d8d 100644 (file)
@@ -24,7 +24,7 @@
 #if ENABLE(VIDEO)
 
 messages -> RemoteMediaPlayerProxy NotRefCounted {
-    PrepareForPlayback(bool privateMode, enum:uint8_t WebCore::MediaPlayerEnums::Preload preload, bool preservesPitch, bool prepareForRendering)
+    PrepareForPlayback(bool privateMode, enum:uint8_t WebCore::MediaPlayerEnums::Preload preload, bool preservesPitch, bool prepareForRendering, WebCore::LayoutRect layoutRect, float videoContentScale) -> (Optional<WebKit::LayerHostingContextID> layerHostingContextId) Async
 
     Load(URL url, Optional<WebKit::SandboxExtension::Handle> sandboxExtension, WebCore::ContentType contentType, String keySystem) -> (struct WebKit::RemoteMediaPlayerConfiguration playerConfiguration) Async
     CancelLoad()
@@ -45,7 +45,6 @@ messages -> RemoteMediaPlayerProxy NotRefCounted {
     SetPreservesPitch(bool preservesPitch)
 
     PrepareForRendering()
-    SetSize(WebCore::IntSize size)
     SetVisible(bool visible)
     SetShouldMaintainAspectRatio(bool maintainAspectRatio)
     SetVideoFullscreenFrame(WebCore::FloatRect rect)
@@ -25,7 +25,6 @@
 
 #pragma once
 
-#include "LayerTreeContext.h"
 #include <wtf/Forward.h>
 #include <wtf/Noncopyable.h>
 #include <wtf/RetainPtr.h>
@@ -40,7 +39,8 @@ class MachSendRight;
 namespace WebKit {
 
 using LayerHostingContextID = uint32_t;
-    
+enum class LayerHostingMode : uint8_t;
+
 struct LayerHostingContextOptions {
 #if PLATFORM(IOS_FAMILY)
     bool canShowWhileLocked { false };
@@ -58,7 +58,9 @@ public:
 #if PLATFORM(MAC)
     static std::unique_ptr<LayerHostingContext> createForExternalPluginHostingProcess();
 #endif
-    
+
+    static RetainPtr<CALayer> createPlatformLayerForHostingContext(LayerHostingContextID);
+
 #endif // HAVE(OUT_OF_PROCESS_LAYER_HOSTING)
 
     LayerHostingContext();
@@ -26,6 +26,8 @@
 #import "config.h"
 #import "LayerHostingContext.h"
 
+#import "LayerTreeContext.h"
+#import <WebCore/WebCoreCALayerExtras.h>
 #import <pal/spi/cg/CoreGraphicsSPI.h>
 #import <pal/spi/cocoa/QuartzCoreSPI.h>
 #import <wtf/MachSendRight.h>
@@ -88,6 +90,12 @@ std::unique_ptr<LayerHostingContext> LayerHostingContext::createForExternalPlugi
     return layerHostingContext;
 }
 #endif
+
+RetainPtr<CALayer> LayerHostingContext::createPlatformLayerForHostingContext(LayerHostingContextID contextID)
+{
+    return [CALayer _web_renderLayerWithContextID:contextID];
+}
+
 #endif // HAVE(OUT_OF_PROCESS_LAYER_HOSTING)
 
 LayerHostingContext::LayerHostingContext()
index f1631ba..5b84efe 100644 (file)
@@ -30,6 +30,7 @@
 
 #import "WKPaymentAuthorizationDelegate.h"
 #import <pal/cocoa/PassKitSoftLink.h>
+#import <wtf/CompletionHandler.h>
 
 @interface WKPaymentAuthorizationControllerDelegate : WKPaymentAuthorizationDelegate <PKPaymentAuthorizationControllerDelegate, PKPaymentAuthorizationControllerPrivateDelegate>
 
index 5680b90..d16cd27 100644 (file)
@@ -29,6 +29,7 @@
 #if ENABLE(NETSCAPE_PLUGIN_API)
 
 #import "LayerHostingContext.h"
+#import "LayerTreeContext.h"
 #import "PluginCreationParameters.h"
 #import "PluginProcess.h"
 #import "PluginProcessProxyMessages.h"
index b73dd81..64f85a6 100644 (file)
@@ -69,6 +69,7 @@ Platform/cg/CGUtilities.cpp
 Platform/classifier/cocoa/ResourceLoadStatisticsClassifierCocoa.cpp
 Platform/classifier/ResourceLoadStatisticsClassifier.cpp
 
+Platform/cocoa/LayerHostingContext.mm
 Platform/cocoa/PaymentAuthorizationPresenter.mm
 Platform/cocoa/PaymentAuthorizationViewController.mm
 Platform/cocoa/SharedMemoryCocoa.cpp
@@ -84,7 +85,6 @@ Platform/ios/PaymentAuthorizationController.mm
 Platform/IPC/cocoa/ConnectionCocoa.mm
 Platform/IPC/cocoa/MachMessage.cpp
 
-Platform/mac/LayerHostingContext.mm
 Platform/mac/MachUtilities.cpp
 Platform/mac/MenuUtilities.mm
 Platform/mac/StringUtilities.mm
index 5eb52c5..2d90eaf 100644 (file)
                1A7E814E1152D2240003695B /* mac */ = {
                        isa = PBXGroup;
                        children = (
-                               BCE0937614FB128B001138D9 /* LayerHostingContext.h */,
-                               BCE0937514FB128B001138D9 /* LayerHostingContext.mm */,
                                1A24B5F011F531E800C38269 /* MachUtilities.cpp */,
                                1A24B5F111F531E800C38269 /* MachUtilities.h */,
                                51933DEB1965EB24008AC3EA /* MenuUtilities.h */,
                4450AEBE1DC3FAAC009943F2 /* cocoa */ = {
                        isa = PBXGroup;
                        children = (
+                               BCE0937614FB128B001138D9 /* LayerHostingContext.h */,
+                               BCE0937514FB128B001138D9 /* LayerHostingContext.mm */,
                                A1798B3D222D97A2000764BD /* PaymentAuthorizationPresenter.h */,
                                A1798B4A222F133A000764BD /* PaymentAuthorizationPresenter.mm */,
                                A1798B3F222D98B6000764BD /* PaymentAuthorizationViewController.h */,
index 7f5f0f2..a1f967d 100644 (file)
 #include <wtf/URL.h>
 #include <wtf/text/CString.h>
 
-#if !RELEASE_LOG_DISABLED
 namespace WebCore {
+#if !RELEASE_LOG_DISABLED
 extern WTFLogChannel LogMedia;
-}
 #endif
+}
 
 namespace WebKit {
 using namespace WebCore;
@@ -89,7 +89,18 @@ MediaPlayerPrivateRemote::~MediaPlayerPrivateRemote()
 
 void MediaPlayerPrivateRemote::prepareForPlayback(bool privateMode, MediaPlayer::Preload preload, bool preservesPitch, bool prepare)
 {
-    connection().send(Messages::RemoteMediaPlayerProxy::PrepareForPlayback(privateMode, preload, preservesPitch, prepare), m_id);
+    auto layoutRect = m_player->playerContentBoxRect();
+    auto scale = m_player->playerContentsScale();
+
+    connection().sendWithAsyncReply(Messages::RemoteMediaPlayerProxy::PrepareForPlayback(privateMode, preload, preservesPitch, prepare, layoutRect, scale), [weakThis = makeWeakPtr(*this), this](auto contextId) mutable {
+        if (!weakThis)
+            return;
+
+        if (!contextId)
+            return;
+
+        m_videoLayer = LayerHostingContext::createPlatformLayerForHostingContext(contextId.value());
+    }, m_id);
 }
 
 void MediaPlayerPrivateRemote::MediaPlayerPrivateRemote::load(const URL& url, const ContentType& contentType, const String& keySystem)
@@ -121,7 +132,7 @@ void MediaPlayerPrivateRemote::MediaPlayerPrivateRemote::load(const URL& url, co
 
     connection().sendWithAsyncReply(Messages::RemoteMediaPlayerProxy::Load(url, sandboxExtensionHandle, contentType, keySystem), [weakThis = makeWeakPtr(*this)](auto&& configuration) {
         if (weakThis)
-            weakThis->m_configuration = configuration;
+            weakThis->m_configuration = WTFMove(configuration);
     }, m_id);
 }
 
@@ -291,6 +302,11 @@ void MediaPlayerPrivateRemote::sizeChanged(WebCore::FloatSize naturalSize)
     m_player->sizeChanged();
 }
 
+void MediaPlayerPrivateRemote::firstVideoFrameAvailable()
+{
+    m_player->firstVideoFrameAvailable();
+}
+
 String MediaPlayerPrivateRemote::engineDescription() const
 {
     return m_configuration.engineDescription;
@@ -346,11 +362,6 @@ void MediaPlayerPrivateRemote::prepareForRendering()
     connection().send(Messages::RemoteMediaPlayerProxy::PrepareForRendering(), m_id);
 }
 
-void MediaPlayerPrivateRemote::setSize(const WebCore::IntSize& size)
-{
-    connection().send(Messages::RemoteMediaPlayerProxy::SetSize(size), m_id);
-}
-
 void MediaPlayerPrivateRemote::setVisible(bool visible)
 {
     connection().send(Messages::RemoteMediaPlayerProxy::SetVisible(visible), m_id);
@@ -480,8 +491,7 @@ void MediaPlayerPrivateRemote::load(MediaStreamPrivate&)
 
 PlatformLayer* MediaPlayerPrivateRemote::platformLayer() const
 {
-    notImplemented();
-    return nullptr;
+    return m_videoLayer.get();
 }
 
 #if PLATFORM(IOS_FAMILY) || (PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE))
index 6eae249..3c972e1 100644 (file)
@@ -78,6 +78,7 @@ public:
     void updateCachedState(RemoteMediaPlayerState&&);
     void characteristicChanged(bool hasAudio, bool hasVideo, WebCore::MediaPlayerEnums::MovieLoadType);
     void sizeChanged(WebCore::FloatSize);
+    void firstVideoFrameAvailable();
 
     void addRemoteAudioTrack(TrackPrivateRemoteIdentifier, TrackPrivateRemoteConfiguration&&);
     void removeRemoteAudioTrack(TrackPrivateRemoteIdentifier);
@@ -123,7 +124,6 @@ private:
 
     bool shouldIgnoreIntrinsicSize() final;
 
-    // Unimplemented
     PlatformLayer* platformLayer() const final;
 
 #if PLATFORM(IOS_FAMILY) || (PLATFORM(MAC) && ENABLE(VIDEO_PRESENTATION_MODE))
@@ -196,7 +196,10 @@ private:
     unsigned long long totalBytes() const final;
     bool didLoadingProgress() const final;
 
-    void setSize(const WebCore::IntSize&) final;
+    // In the Cocoa WebKit port, MediaPlayerPrivateAVFoundationObjC::setSize() does nothing,
+    // so the Web process does not need to send IPC messages to call it in the GPU process.
+    // Other WebKit ports may need to do that.
+    void setSize(const WebCore::IntSize&) final { }
 
     void paint(WebCore::GraphicsContext&, const WebCore::FloatRect&) final;
 
@@ -316,6 +319,7 @@ private:
 
     WebCore::MediaPlayer* m_player { nullptr };
     RefPtr<WebCore::PlatformMediaResourceLoader> m_mediaResourceLoader;
+    RetainPtr<PlatformLayer> m_videoLayer;
     RemoteMediaPlayerManager& m_manager;
     WebCore::MediaPlayerEnums::MediaEngineIdentifier m_remoteEngineIdentifier;
     MediaPlayerPrivateRemoteIdentifier m_id;
index b7c04f1..cadd88a 100644 (file)
@@ -293,13 +293,13 @@ void RemoteMediaPlayerManager::engineFailedToLoad(MediaPlayerPrivateRemoteIdenti
 
 void RemoteMediaPlayerManager::characteristicChanged(MediaPlayerPrivateRemoteIdentifier id, bool hasAudio, bool hasVideo, WebCore::MediaPlayerEnums::MovieLoadType loadType)
 {
-    if (auto player = m_players.get(id))
+    if (const auto& player = m_players.get(id))
         player->characteristicChanged(hasAudio, hasVideo, loadType);
 }
 
 void RemoteMediaPlayerManager::sizeChanged(MediaPlayerPrivateRemoteIdentifier id, WebCore::FloatSize naturalSize)
 {
-    if (auto player = m_players.get(id))
+    if (const auto& player = m_players.get(id))
         player->sizeChanged(naturalSize);
 }
 
@@ -323,22 +323,28 @@ void RemoteMediaPlayerManager::remoteAudioTrackConfigurationChanged(MediaPlayerP
 
 void RemoteMediaPlayerManager::addRemoteVideoTrack(MediaPlayerPrivateRemoteIdentifier playerID, TrackPrivateRemoteIdentifier trackID, TrackPrivateRemoteConfiguration&& configuration)
 {
-    if (auto player = m_players.get(playerID))
+    if (const auto& player = m_players.get(playerID))
         player->addRemoteVideoTrack(trackID, WTFMove(configuration));
 }
 
 void RemoteMediaPlayerManager::removeRemoteVideoTrack(MediaPlayerPrivateRemoteIdentifier playerID, TrackPrivateRemoteIdentifier trackID)
 {
-    if (auto player = m_players.get(playerID))
+    if (const auto& player = m_players.get(playerID))
         player->removeRemoteVideoTrack(trackID);
 }
 
 void RemoteMediaPlayerManager::remoteVideoTrackConfigurationChanged(MediaPlayerPrivateRemoteIdentifier playerID, TrackPrivateRemoteIdentifier trackID, TrackPrivateRemoteConfiguration&& configuration)
 {
-    if (auto player = m_players.get(playerID))
+    if (const auto& player = m_players.get(playerID))
         player->remoteVideoTrackConfigurationChanged(trackID, WTFMove(configuration));
 }
 
+void RemoteMediaPlayerManager::firstVideoFrameAvailable(WebKit::MediaPlayerPrivateRemoteIdentifier id)
+{
+    if (const auto& player = m_players.get(id))
+        player->firstVideoFrameAvailable();
+}
+
 void RemoteMediaPlayerManager::requestResource(MediaPlayerPrivateRemoteIdentifier id, RemoteMediaResourceIdentifier remoteMediaResourceIdentifier, ResourceRequest&& request, PlatformMediaResourceLoader::LoadOptions options, CompletionHandler<void()>&& completionHandler)
 {
     if (const auto& player = m_players.get(id))
@@ -364,7 +370,7 @@ void RemoteMediaPlayerManager::updatePreferences(const Settings& settings)
 
 void RemoteMediaPlayerManager::updateCachedState(MediaPlayerPrivateRemoteIdentifier id, RemoteMediaPlayerState&& state)
 {
-    if (auto player = m_players.get(id))
+    if (const auto& player = m_players.get(id))
         player->updateCachedState(WTFMove(state));
 }
 
index f43f9dc..817634f 100644 (file)
@@ -93,6 +93,7 @@ private:
     void addRemoteAudioTrack(MediaPlayerPrivateRemoteIdentifier, TrackPrivateRemoteIdentifier, TrackPrivateRemoteConfiguration&&);
     void removeRemoteAudioTrack(MediaPlayerPrivateRemoteIdentifier, TrackPrivateRemoteIdentifier);
     void remoteAudioTrackConfigurationChanged(MediaPlayerPrivateRemoteIdentifier, TrackPrivateRemoteIdentifier, TrackPrivateRemoteConfiguration&&);
+    void firstVideoFrameAvailable(WebKit::MediaPlayerPrivateRemoteIdentifier);
 
     void addRemoteVideoTrack(MediaPlayerPrivateRemoteIdentifier, TrackPrivateRemoteIdentifier, TrackPrivateRemoteConfiguration&&);
     void removeRemoteVideoTrack(MediaPlayerPrivateRemoteIdentifier, TrackPrivateRemoteIdentifier);
index 32ea7d0..37c6845 100644 (file)
@@ -26,6 +26,7 @@
 messages -> RemoteMediaPlayerManager NotRefCounted {
     NetworkStateChanged(WebKit::MediaPlayerPrivateRemoteIdentifier id, struct WebKit::RemoteMediaPlayerState state)
     ReadyStateChanged(WebKit::MediaPlayerPrivateRemoteIdentifier id, struct WebKit::RemoteMediaPlayerState state)
+    FirstVideoFrameAvailable(WebKit::MediaPlayerPrivateRemoteIdentifier id)
     VolumeChanged(WebKit::MediaPlayerPrivateRemoteIdentifier id, double volume)
     MuteChanged(WebKit::MediaPlayerPrivateRemoteIdentifier id, bool mute)
     TimeChanged(WebKit::MediaPlayerPrivateRemoteIdentifier id, struct WebKit::RemoteMediaPlayerState state)
index 977b8ed..617d7d7 100644 (file)
@@ -48,6 +48,7 @@
 
 #if PLATFORM(COCOA)
 #include "LayerHostingContext.h"
+#include "LayerTreeContext.h"
 #endif
 
 namespace WebKit {