Implement MediaRecorder backend in GPUProcess
authoryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 7 Jan 2020 17:08:02 +0000 (17:08 +0000)
committeryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Tue, 7 Jan 2020 17:08:02 +0000 (17:08 +0000)
https://bugs.webkit.org/show_bug.cgi?id=205802

Reviewed by Eric Carlson.

Source/WebCore:

Add a page provider to create MediaRecorderPrivate implementations.
This is used by WebKit layer to implement this in GPUProcess.

Update MediaRecorderPrivate by adding an error callback that is used to surface errors as MediaRecorder error events.

Covered by existing tests as MediaRecorder implementation in WebKitTestRunner will use GPUProcess.

* Headers.cmake:
* Modules/mediarecorder/MediaRecorder.cpp:
(WebCore::MediaRecorder::create):
(WebCore::MediaRecorder::createMediaRecorderPrivate):
* Modules/mediarecorder/MediaRecorder.h:
* Modules/mediarecorder/MediaRecorderProvider.cpp: Added.
(WebCore::MediaRecorderProvider::createMediaRecorderPrivate):
* Modules/mediarecorder/MediaRecorderProvider.h: Added.
* Sources.txt:
* WebCore.xcodeproj/project.pbxproj:
* loader/EmptyClients.cpp:
(WebCore::pageConfigurationWithEmptyClients):
* page/Page.cpp:
(WebCore::Page::Page):
* page/Page.h:
(WebCore::Page::mediaRecorderProvider):
* page/PageConfiguration.cpp:
(WebCore::PageConfiguration::PageConfiguration):
* page/PageConfiguration.h:
* platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
* platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
(WebCore::MediaRecorderPrivateWriter::create):
* platform/mediastream/MediaStreamPrivate.h:
* platform/mediastream/MediaStreamTrackPrivate.h:
* testing/Internals.cpp:
(WebCore::Internals::resetToConsistentState):
(WebCore::Internals::setUseGPUProcessForWebRTC):

Source/WebKit:

Add support for sending audio/video tracks to record from WebProcess to GPUProcess.
Add a MediaRecorderPrivate implementation that supports sending one audio track and/or one video track to GPUProcess
and stopping/fetching data from the remote recorder in GPUProcess.

In GPUProcess, implement the remote recorder using the existing WebCore recorder writer.

* CMakeLists.txt:
* DerivedSources-input.xcfilelist:
* DerivedSources-output.xcfilelist:
* DerivedSources.make:
* GPUProcess/GPUConnectionToWebProcess.cpp:
(WebKit::GPUConnectionToWebProcess::userMediaCaptureManagerProxy):
(WebKit::GPUConnectionToWebProcess::mediaRecorderManager):
(WebKit::GPUConnectionToWebProcess::didReceiveMessage):
* GPUProcess/GPUConnectionToWebProcess.h:
* GPUProcess/mac/com.apple.WebKit.GPUProcess.sb.in:
* GPUProcess/webrtc/RemoteMediaRecorder.cpp: Added.
(WebKit::RemoteMediaRecorder::create):
(WebKit::RemoteMediaRecorder::RemoteMediaRecorder):
(WebKit::RemoteMediaRecorder::~RemoteMediaRecorder):
(WebKit::RemoteMediaRecorder::storage):
(WebKit::RemoteMediaRecorder::audioSamplesStorageChanged):
(WebKit::RemoteMediaRecorder::audioSamplesAvailable):
(WebKit::RemoteMediaRecorder::videoSampleAvailable):
(WebKit::RemoteMediaRecorder::fetchData):
(WebKit::RemoteMediaRecorder::stopRecording):
* GPUProcess/webrtc/RemoteMediaRecorder.h: Added.
* GPUProcess/webrtc/RemoteMediaRecorder.messages.in: Added.
* GPUProcess/webrtc/RemoteMediaRecorderManager.cpp: Added.
(WebKit::RemoteMediaRecorderManager::RemoteMediaRecorderManager):
(WebKit::RemoteMediaRecorderManager::~RemoteMediaRecorderManager):
(WebKit::RemoteMediaRecorderManager::didReceiveRemoteMediaRecorderMessage):
(WebKit::RemoteMediaRecorderManager::createRecorder):
(WebKit::RemoteMediaRecorderManager::releaseRecorder):
* GPUProcess/webrtc/RemoteMediaRecorderManager.h: Added.
(WebKit::RemoteMediaRecorderManager::didReceiveMessageFromWebProcess):
* GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in: Added.
* Scripts/webkit/messages.py:
* Sources.txt:
* SourcesCocoa.txt:
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/GPU/media/MediaRecorderProvider.cpp: Added.
(WebCore::MediaRecorderProvider::createMediaRecorderPrivate):
* WebProcess/GPU/media/MediaRecorderProvider.h: Added.
* WebProcess/GPU/webrtc/MediaRecorderIdentifier.h: Added.
* WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp: Added.
(WebKit::MediaRecorderPrivate::MediaRecorderPrivate):
(WebKit::MediaRecorderPrivate::~MediaRecorderPrivate):
(WebKit::MediaRecorderPrivate::sampleBufferUpdated):
(WebKit::MediaRecorderPrivate::audioSamplesAvailable):
(WebKit::MediaRecorderPrivate::storageChanged):
(WebKit::MediaRecorderPrivate::fetchData):
(WebKit::MediaRecorderPrivate::stopRecording):
* WebProcess/GPU/webrtc/MediaRecorderPrivate.h: Added.
* WebProcess/GPU/webrtc/MediaRecorderProvider.cpp: Added.
(WebKit::MediaRecorderProvider::createMediaRecorderPrivate):
* WebProcess/GPU/webrtc/MediaRecorderProvider.h: Added.
* WebProcess/WebPage/WebPage.cpp:
(WebKit::m_overriddenMediaType):

Source/WebKitLegacy/mac:

* WebView/WebView.mm:
(-[WebView _commonInitializationWithFrameName:groupName:]):
(-[WebView initSimpleHTMLDocumentWithStyle:frame:preferences:groupName:]):

Source/WebKitLegacy/win:

* WebView.cpp:
(WebView::initWithFrame):

LayoutTests:

* http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable.html:
Do not use GPUProcess for this test until canvas buffers are IOSurfaces.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254132 268f45cc-cd09-0410-ab3c-d52691b4dbfc

54 files changed:
LayoutTests/ChangeLog
LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable.html
Source/WebCore/ChangeLog
Source/WebCore/Headers.cmake
Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp
Source/WebCore/Modules/mediarecorder/MediaRecorder.h
Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp [new file with mode: 0644]
Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h [new file with mode: 0644]
Source/WebCore/Sources.txt
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/loader/EmptyClients.cpp
Source/WebCore/page/Page.cpp
Source/WebCore/page/Page.h
Source/WebCore/page/PageConfiguration.cpp
Source/WebCore/page/PageConfiguration.h
Source/WebCore/platform/mediarecorder/MediaRecorderPrivate.h
Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp
Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.h
Source/WebCore/platform/mediarecorder/MediaRecorderPrivateMock.cpp
Source/WebCore/platform/mediarecorder/MediaRecorderPrivateMock.h
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm
Source/WebCore/platform/mediastream/MediaStreamPrivate.h
Source/WebCore/platform/mediastream/MediaStreamTrackPrivate.h
Source/WebCore/testing/Internals.cpp
Source/WebKit/CMakeLists.txt
Source/WebKit/ChangeLog
Source/WebKit/DerivedSources-input.xcfilelist
Source/WebKit/DerivedSources-output.xcfilelist
Source/WebKit/DerivedSources.make
Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.cpp [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.h [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in [new file with mode: 0644]
Source/WebKit/Scripts/webkit/messages.py
Source/WebKit/Sources.txt
Source/WebKit/SourcesCocoa.txt
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/GPU/media/MediaRecorderProvider.cpp [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/media/MediaRecorderProvider.h [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderIdentifier.h [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.cpp [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.h [new file with mode: 0644]
Source/WebKit/WebProcess/WebPage/WebPage.cpp
Source/WebKitLegacy/mac/ChangeLog
Source/WebKitLegacy/mac/WebView/WebView.mm
Source/WebKitLegacy/win/ChangeLog
Source/WebKitLegacy/win/WebView.cpp

index 5648697..aad1679 100644 (file)
@@ -1,3 +1,13 @@
+2020-01-07  youenn fablet  <youenn@apple.com>
+
+        Implement MediaRecorder backend in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=205802
+
+        Reviewed by Eric Carlson.
+
+        * http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable.html:
+        Do not use GPUProcess for this test until canvas buffers are IOSurfaces.
+
 2020-01-07  Charlie Turner  <mail@charles.plus>
 
         [GTK][EME] ClearKey encrypted media gardening
index 3c5f539..44b6a2f 100644 (file)
     </canvas>
 </div>
 <script>
+    // This test is not passing with GPUProcess as long as capture frames are not IOSurfaced backed.
+    if (window.internals)
+        window.internals.setUseGPUProcessForWebRTC(false);
+
     var context;
     var drawStartTime;
 
 
 </script>
 </body>
-</html>
\ No newline at end of file
+</html>
index 1e0d183..96d75ad 100644 (file)
@@ -1,3 +1,45 @@
+2020-01-07  youenn fablet  <youenn@apple.com>
+
+        Implement MediaRecorder backend in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=205802
+
+        Reviewed by Eric Carlson.
+
+        Add a page provider to create MediaRecorderPrivate implementations.
+        This is used by WebKit layer to implement this in GPUProcess.
+
+        Update MediaRecorderPrivate by adding an error callback that is used to surface errors as MediaRecorder error events.
+
+        Covered by existing tests as MediaRecorder implementation in WebKitTestRunner will use GPUProcess.
+
+        * Headers.cmake:
+        * Modules/mediarecorder/MediaRecorder.cpp:
+        (WebCore::MediaRecorder::create):
+        (WebCore::MediaRecorder::createMediaRecorderPrivate):
+        * Modules/mediarecorder/MediaRecorder.h:
+        * Modules/mediarecorder/MediaRecorderProvider.cpp: Added.
+        (WebCore::MediaRecorderProvider::createMediaRecorderPrivate):
+        * Modules/mediarecorder/MediaRecorderProvider.h: Added.
+        * Sources.txt:
+        * WebCore.xcodeproj/project.pbxproj:
+        * loader/EmptyClients.cpp:
+        (WebCore::pageConfigurationWithEmptyClients):
+        * page/Page.cpp:
+        (WebCore::Page::Page):
+        * page/Page.h:
+        (WebCore::Page::mediaRecorderProvider):
+        * page/PageConfiguration.cpp:
+        (WebCore::PageConfiguration::PageConfiguration):
+        * page/PageConfiguration.h:
+        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
+        * platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
+        (WebCore::MediaRecorderPrivateWriter::create):
+        * platform/mediastream/MediaStreamPrivate.h:
+        * platform/mediastream/MediaStreamTrackPrivate.h:
+        * testing/Internals.cpp:
+        (WebCore::Internals::resetToConsistentState):
+        (WebCore::Internals::setUseGPUProcessForWebRTC):
+
 2020-01-07  Rob Buis  <rbuis@igalia.com>
 
         Remove loader/win
index 46619c6..812ed46 100644 (file)
@@ -80,6 +80,8 @@ set(WebCore_PRIVATE_FRAMEWORK_HEADERS
     Modules/indexeddb/shared/IDBResultData.h
     Modules/indexeddb/shared/IDBTransactionInfo.h
 
+    Modules/mediarecorder/MediaRecorderProvider.h
+
     Modules/mediasession/MediaSessionEvents.h
     Modules/mediasession/MediaSessionMetadata.h
     Modules/mediasession/WebMediaSessionManager.h
@@ -1155,6 +1157,8 @@ set(WebCore_PRIVATE_FRAMEWORK_HEADERS
     platform/graphics/transforms/TransformOperations.h
     platform/graphics/transforms/TransformationMatrix.h
 
+    platform/mediarecorder/MediaRecorderPrivate.h
+
     platform/mediastream/CaptureDevice.h
     platform/mediastream/CaptureDeviceManager.h
     platform/mediastream/MediaConstraints.h
index 991ea2a..44c3187 100644 (file)
@@ -34,6 +34,8 @@
 #include "EventNames.h"
 #include "MediaRecorderErrorEvent.h"
 #include "MediaRecorderPrivate.h"
+#include "MediaRecorderProvider.h"
+#include "Page.h"
 #include "SharedBuffer.h"
 #include "WindowEventLoop.h"
 #include <wtf/IsoMallocInlines.h>
@@ -50,11 +52,14 @@ creatorFunction MediaRecorder::m_customCreator = nullptr;
 
 ExceptionOr<Ref<MediaRecorder>> MediaRecorder::create(Document& document, Ref<MediaStream>&& stream, Options&& options)
 {
-    auto privateInstance = MediaRecorder::getPrivateImpl(stream->privateStream());
+    auto privateInstance = MediaRecorder::createMediaRecorderPrivate(document, stream->privateStream());
     if (!privateInstance)
         return Exception { NotSupportedError, "The MediaRecorder is unsupported on this platform"_s };
     auto recorder = adoptRef(*new MediaRecorder(document, WTFMove(stream), WTFMove(privateInstance), WTFMove(options)));
     recorder->suspendIfNeeded();
+    recorder->m_private->setErrorCallback([recorder = recorder.copyRef()](auto&& exception) mutable {
+        recorder->dispatchError(WTFMove(*exception));
+    });
     return recorder;
 }
 
@@ -63,17 +68,16 @@ void MediaRecorder::setCustomPrivateRecorderCreator(creatorFunction creator)
     m_customCreator = creator;
 }
 
-std::unique_ptr<MediaRecorderPrivate> MediaRecorder::getPrivateImpl(const MediaStreamPrivate& stream)
+std::unique_ptr<MediaRecorderPrivate> MediaRecorder::createMediaRecorderPrivate(Document& document, const MediaStreamPrivate& stream)
 {
     if (m_customCreator)
         return m_customCreator();
     
-#if PLATFORM(COCOA)
-    return MediaRecorderPrivateAVFImpl::create(stream);
-#else
-    UNUSED_PARAM(stream);
-    return nullptr;
-#endif
+    auto* page = document.page();
+    if (!page)
+        return nullptr;
+
+    return page->mediaRecorderProvider().createMediaRecorderPrivate(stream);
 }
 
 MediaRecorder::MediaRecorder(Document& document, Ref<MediaStream>&& stream, std::unique_ptr<MediaRecorderPrivate>&& privateImpl, Options&& option)
@@ -182,11 +186,17 @@ void MediaRecorder::didAddOrRemoveTrack()
         if (!m_isActive || state() == RecordingState::Inactive)
             return;
         stopRecordingInternal();
-        auto event = MediaRecorderErrorEvent::create(eventNames().errorEvent, Exception { UnknownError, "Track cannot be added to or removed from the MediaStream while recording is happening"_s });
-        dispatchEvent(WTFMove(event));
+        dispatchError(Exception { UnknownError, "Track cannot be added to or removed from the MediaStream while recording is happening"_s });
     });
 }
 
+void MediaRecorder::dispatchError(Exception&& exception)
+{
+    if (!m_isActive)
+        return;
+    dispatchEvent(MediaRecorderErrorEvent::create(eventNames().errorEvent, WTFMove(exception)));
+}
+
 void MediaRecorder::trackEnded(MediaStreamTrackPrivate&)
 {
     auto position = m_tracks.findMatching([](auto& track) {
index eb6f96a..e90b99b 100644 (file)
@@ -71,11 +71,11 @@ public:
     
     ExceptionOr<void> startRecording(Optional<int>);
     ExceptionOr<void> stopRecording();
-    
+
 private:
     MediaRecorder(Document&, Ref<MediaStream>&&, std::unique_ptr<MediaRecorderPrivate>&&, Options&& = { });
 
-    static std::unique_ptr<MediaRecorderPrivate> getPrivateImpl(const MediaStreamPrivate&);
+    static std::unique_ptr<MediaRecorderPrivate> createMediaRecorderPrivate(Document&, const MediaStreamPrivate&);
     
     Document* document() const;
 
@@ -91,7 +91,9 @@ private:
     const char* activeDOMObjectName() const final;
     
     void stopRecordingInternal();
-    
+
+    void dispatchError(Exception&&);
+
     // MediaStream::Observer
     void didAddOrRemoveTrack() final;
     
diff --git a/Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp b/Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp
new file mode 100644 (file)
index 0000000..f64fccc
--- /dev/null
@@ -0,0 +1,49 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "MediaRecorderProvider.h"
+
+#include "MediaRecorderPrivate.h"
+
+#if PLATFORM(COCOA)
+#include "MediaRecorderPrivateAVFImpl.h"
+#endif
+
+namespace WebCore {
+
+#if ENABLE(MEDIA_STREAM)
+std::unique_ptr<MediaRecorderPrivate> MediaRecorderProvider::createMediaRecorderPrivate(const MediaStreamPrivate& stream)
+{
+#if PLATFORM(COCOA)
+    return MediaRecorderPrivateAVFImpl::create(stream);
+#else
+    UNUSED_PARAM(stream);
+    return nullptr;
+#endif
+}
+#endif
+
+}
diff --git a/Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h b/Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h
new file mode 100644 (file)
index 0000000..4a93af4
--- /dev/null
@@ -0,0 +1,49 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+namespace WebCore {
+
+class MediaRecorderPrivate;
+class MediaStreamPrivate;
+
+class WEBCORE_EXPORT MediaRecorderProvider {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    MediaRecorderProvider() = default;
+    virtual ~MediaRecorderProvider() = default;
+
+#if ENABLE(MEDIA_STREAM)
+    virtual std::unique_ptr<MediaRecorderPrivate> createMediaRecorderPrivate(const MediaStreamPrivate&);
+#endif
+
+    void setUseGPUProcess(bool value) { m_useGPUProcess = value; }
+
+protected:
+    bool m_useGPUProcess { false };
+};
+
+}
index 60bbbfd..4112155 100644 (file)
@@ -151,6 +151,7 @@ Modules/mediacontrols/MediaControlsHost.cpp
 
 Modules/mediarecorder/BlobEvent.cpp
 Modules/mediarecorder/MediaRecorder.cpp
+Modules/mediarecorder/MediaRecorderProvider.cpp
 Modules/mediarecorder/MediaRecorderErrorEvent.cpp
 
 Modules/mediasession/HTMLMediaElementMediaSession.cpp
index ec4084d..4514af6 100644 (file)
                417612B01E3A994000C3D81D /* LibWebRTCMediaEndpoint.h in Headers */ = {isa = PBXBuildFile; fileRef = 417612AC1E3A993B00C3D81D /* LibWebRTCMediaEndpoint.h */; };
                417612B11E3A994000C3D81D /* LibWebRTCPeerConnectionBackend.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 417612AD1E3A993B00C3D81D /* LibWebRTCPeerConnectionBackend.cpp */; };
                417612B21E3A994000C3D81D /* LibWebRTCPeerConnectionBackend.h in Headers */ = {isa = PBXBuildFile; fileRef = 417612AE1E3A993B00C3D81D /* LibWebRTCPeerConnectionBackend.h */; };
+               4176E89623C3537B003E83FE /* MediaRecorderProvider.h in Headers */ = {isa = PBXBuildFile; fileRef = 4176E88F23C348D2003E83FE /* MediaRecorderProvider.h */; settings = {ATTRIBUTES = (Private, ); }; };
                417DA6D913734E6E007C57FB /* Internals.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 417DA4CF13734326007C57FB /* Internals.cpp */; };
                417DA6DA13734E6E007C57FB /* Internals.h in Headers */ = {isa = PBXBuildFile; fileRef = 417DA4CE13734326007C57FB /* Internals.h */; };
                417DA71D13735DFA007C57FB /* JSInternals.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 417DA71B13735DFA007C57FB /* JSInternals.cpp */; };
                4BDEA32C218033EB0052DFCD /* JSWorkletGlobalScopeBase.h in Headers */ = {isa = PBXBuildFile; fileRef = 4BDEA32B218033EB0052DFCD /* JSWorkletGlobalScopeBase.h */; };
                4D3B00AB215D69A70076B983 /* MediaRecorder.h in Headers */ = {isa = PBXBuildFile; fileRef = 4D3B00A9215D69A70076B983 /* MediaRecorder.h */; };
                4D3B00AF215D6A690076B983 /* BlobEvent.h in Headers */ = {isa = PBXBuildFile; fileRef = 4D3B00AD215D6A690076B983 /* BlobEvent.h */; };
-               4D3B5016217E58B700665DB1 /* MediaRecorderPrivate.h in Headers */ = {isa = PBXBuildFile; fileRef = 4D3B5014217E58B700665DB1 /* MediaRecorderPrivate.h */; };
+               4D3B5016217E58B700665DB1 /* MediaRecorderPrivate.h in Headers */ = {isa = PBXBuildFile; fileRef = 4D3B5014217E58B700665DB1 /* MediaRecorderPrivate.h */; settings = {ATTRIBUTES = (Private, ); }; };
                4D73F946218BC5FA003A3ED6 /* MediaRecorderPrivateAVFImpl.h in Headers */ = {isa = PBXBuildFile; fileRef = 4D73F944218BC5FA003A3ED6 /* MediaRecorderPrivateAVFImpl.h */; };
-               4D73F94E218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.h in Headers */ = {isa = PBXBuildFile; fileRef = 4D73F94C218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.h */; };
+               4D73F94E218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.h in Headers */ = {isa = PBXBuildFile; fileRef = 4D73F94C218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.h */; settings = {ATTRIBUTES = (Private, ); }; };
                4DB7130D216ECB4D0096A4DD /* MediaRecorderErrorEvent.h in Headers */ = {isa = PBXBuildFile; fileRef = 4DB7130C216EC2BD0096A4DD /* MediaRecorderErrorEvent.h */; };
                4E1959220A39DABA00220FE5 /* MediaFeatureNames.h in Headers */ = {isa = PBXBuildFile; fileRef = 4E1959200A39DABA00220FE5 /* MediaFeatureNames.h */; settings = {ATTRIBUTES = (Private, ); }; };
                4E19592A0A39DACC00220FE5 /* MediaQuery.h in Headers */ = {isa = PBXBuildFile; fileRef = 4E1959240A39DACC00220FE5 /* MediaQuery.h */; settings = {ATTRIBUTES = (Private, ); }; };
                417612AE1E3A993B00C3D81D /* LibWebRTCPeerConnectionBackend.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = LibWebRTCPeerConnectionBackend.h; path = libwebrtc/LibWebRTCPeerConnectionBackend.h; sourceTree = "<group>"; };
                4176900322FCD8F200B1576D /* MediaSourceRegistry.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaSourceRegistry.h; sourceTree = "<group>"; };
                4176900422FCD8F200B1576D /* MediaSourceRegistry.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MediaSourceRegistry.cpp; sourceTree = "<group>"; };
+               4176E88F23C348D2003E83FE /* MediaRecorderProvider.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaRecorderProvider.h; sourceTree = "<group>"; };
+               4176E89123C34B75003E83FE /* MediaRecorderProvider.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = MediaRecorderProvider.cpp; sourceTree = "<group>"; };
                4177F51C2382544000C04486 /* BaseAudioSharedUnit.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = BaseAudioSharedUnit.cpp; sourceTree = "<group>"; };
                4177F51E2382545E00C04486 /* BaseAudioSharedUnit.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = BaseAudioSharedUnit.h; sourceTree = "<group>"; };
                417DA4CE13734326007C57FB /* Internals.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = Internals.h; sourceTree = "<group>"; };
                                4DB7130E216EEBAE0096A4DD /* MediaRecorderErrorEvent.cpp */,
                                4DB7130C216EC2BD0096A4DD /* MediaRecorderErrorEvent.h */,
                                4DB7130A216EC2BC0096A4DD /* MediaRecorderErrorEvent.idl */,
+                               4176E89123C34B75003E83FE /* MediaRecorderProvider.cpp */,
+                               4176E88F23C348D2003E83FE /* MediaRecorderProvider.h */,
                        );
                        path = mediarecorder;
                        sourceTree = "<group>";
                                4D3B5016217E58B700665DB1 /* MediaRecorderPrivate.h in Headers */,
                                4D73F946218BC5FA003A3ED6 /* MediaRecorderPrivateAVFImpl.h in Headers */,
                                4D73F94E218C4A87003A3ED6 /* MediaRecorderPrivateWriterCocoa.h in Headers */,
+                               4176E89623C3537B003E83FE /* MediaRecorderProvider.h in Headers */,
                                C90843D01B18E47D00B68564 /* MediaRemoteControls.h in Headers */,
                                CD8ACA8F1D23971900ECC59E /* MediaRemoteSoftLink.h in Headers */,
                                CEEFCD7A19DB31F7003876D7 /* MediaResourceLoader.h in Headers */,
index 84e4e18..cb51f36 100644 (file)
@@ -53,6 +53,8 @@
 #include "IDBConnectionToServer.h"
 #include "InspectorClient.h"
 #include "LibWebRTCProvider.h"
+#include "MediaRecorderPrivate.h"
+#include "MediaRecorderProvider.h"
 #include "NetworkStorageSession.h"
 #include "Page.h"
 #include "PageConfiguration.h"
@@ -571,10 +573,19 @@ Ref<StorageNamespace> EmptyStorageNamespaceProvider::createTransientLocalStorage
     return adoptRef(*new EmptyStorageNamespace { sessionID });
 }
 
-class EmptyStorageSessionProvider : public StorageSessionProvider {
+class EmptyStorageSessionProvider final : public StorageSessionProvider {
     NetworkStorageSession* storageSession() const final { return nullptr; }
 };
 
+class EmptyMediaRecorderProvider final : public MediaRecorderProvider {
+public:
+    EmptyMediaRecorderProvider() = default;
+private:
+#if ENABLE(MEDIA_STREAM)
+    std::unique_ptr<MediaRecorderPrivate> createMediaRecorderPrivate(const MediaStreamPrivate&) final { return nullptr; }
+#endif
+};
+
 PageConfiguration pageConfigurationWithEmptyClients(PAL::SessionID sessionID)
 {
     PageConfiguration pageConfiguration {
@@ -585,7 +596,8 @@ PageConfiguration pageConfigurationWithEmptyClients(PAL::SessionID sessionID)
         CacheStorageProvider::create(),
         adoptRef(*new EmptyBackForwardClient),
         CookieJar::create(adoptRef(*new EmptyStorageSessionProvider)),
-        makeUniqueRef<EmptyProgressTrackerClient>()
+        makeUniqueRef<EmptyProgressTrackerClient>(),
+        makeUniqueRef<EmptyMediaRecorderProvider>()
     };
 
     static NeverDestroyed<EmptyChromeClient> dummyChromeClient;
index 63c0534..010279f 100644 (file)
@@ -246,6 +246,7 @@ Page::Page(PageConfiguration&& pageConfiguration)
 #if ENABLE(SPEECH_SYNTHESIS)
     , m_speechSynthesisClient(WTFMove(pageConfiguration.speechSynthesisClient))
 #endif
+    , m_mediaRecorderProvider((WTFMove(pageConfiguration.mediaRecorderProvider)))
     , m_libWebRTCProvider(WTFMove(pageConfiguration.libWebRTCProvider))
     , m_verticalScrollElasticity(ScrollElasticityAllowed)
     , m_horizontalScrollElasticity(ScrollElasticityAllowed)
index 2c3f7c3..e1f27cf 100644 (file)
@@ -110,6 +110,7 @@ class LibWebRTCProvider;
 class LowPowerModeNotifier;
 class MediaCanStartListener;
 class MediaPlaybackTarget;
+class MediaRecorderProvider;
 class PageConfiguration;
 class PageConsoleClient;
 class PageDebuggable;
@@ -584,6 +585,7 @@ public:
     DatabaseProvider& databaseProvider() { return m_databaseProvider; }
     CacheStorageProvider& cacheStorageProvider() { return m_cacheStorageProvider; }
     SocketProvider& socketProvider() { return m_socketProvider; }
+    MediaRecorderProvider& mediaRecorderProvider() { return m_mediaRecorderProvider; }
     CookieJar& cookieJar() { return m_cookieJar.get(); }
 
     StorageNamespaceProvider& storageNamespaceProvider() { return m_storageNamespaceProvider.get(); }
@@ -795,6 +797,7 @@ private:
     std::unique_ptr<SpeechSynthesisClient> m_speechSynthesisClient;
 #endif
 
+    UniqueRef<MediaRecorderProvider> m_mediaRecorderProvider;
     UniqueRef<LibWebRTCProvider> m_libWebRTCProvider;
     RTCController m_rtcController;
 
@@ -920,7 +923,6 @@ private:
     Ref<UserContentProvider> m_userContentProvider;
     Ref<VisitedLinkStore> m_visitedLinkStore;
     RefPtr<WheelEventTestMonitor> m_wheelEventTestMonitor;
-
     HashSet<ActivityStateChangeObserver*> m_activityStateChangeObservers;
 
 #if ENABLE(RESOURCE_USAGE)
index ef877a5..338f4e7 100644 (file)
@@ -36,6 +36,7 @@
 #include "DragClient.h"
 #include "EditorClient.h"
 #include "LibWebRTCProvider.h"
+#include "MediaRecorderProvider.h"
 #include "PerformanceLoggingClient.h"
 #include "PlugInClient.h"
 #include "PluginInfoProvider.h"
@@ -55,7 +56,7 @@
 
 namespace WebCore {
 
-PageConfiguration::PageConfiguration(PAL::SessionID sessionID, UniqueRef<EditorClient>&& editorClient, Ref<SocketProvider>&& socketProvider, UniqueRef<LibWebRTCProvider>&& libWebRTCProvider, Ref<CacheStorageProvider>&& cacheStorageProvider, Ref<BackForwardClient>&& backForwardClient, Ref<CookieJar>&& cookieJar, UniqueRef<ProgressTrackerClient>&& progressTrackerClient)
+PageConfiguration::PageConfiguration(PAL::SessionID sessionID, UniqueRef<EditorClient>&& editorClient, Ref<SocketProvider>&& socketProvider, UniqueRef<LibWebRTCProvider>&& libWebRTCProvider, Ref<CacheStorageProvider>&& cacheStorageProvider, Ref<BackForwardClient>&& backForwardClient, Ref<CookieJar>&& cookieJar, UniqueRef<ProgressTrackerClient>&& progressTrackerClient, UniqueRef<MediaRecorderProvider>&& mediaRecorderProvider)
     : sessionID(sessionID)
     , editorClient(WTFMove(editorClient))
     , socketProvider(WTFMove(socketProvider))
@@ -64,6 +65,7 @@ PageConfiguration::PageConfiguration(PAL::SessionID sessionID, UniqueRef<EditorC
     , backForwardClient(WTFMove(backForwardClient))
     , cookieJar(WTFMove(cookieJar))
     , cacheStorageProvider(WTFMove(cacheStorageProvider))
+    , mediaRecorderProvider(WTFMove(mediaRecorderProvider))
 {
 }
 
index 191a02b..3a8ed33 100644 (file)
@@ -52,6 +52,7 @@ class EditorClient;
 class FrameLoaderClient;
 class InspectorClient;
 class LibWebRTCProvider;
+class MediaRecorderProvider;
 class PaymentCoordinatorClient;
 class PerformanceLoggingClient;
 class PlugInClient;
@@ -68,7 +69,7 @@ class SpeechSynthesisClient;
 class PageConfiguration {
     WTF_MAKE_NONCOPYABLE(PageConfiguration); WTF_MAKE_FAST_ALLOCATED;
 public:
-    WEBCORE_EXPORT PageConfiguration(PAL::SessionID, UniqueRef<EditorClient>&&, Ref<SocketProvider>&&, UniqueRef<LibWebRTCProvider>&&, Ref<CacheStorageProvider>&&, Ref<BackForwardClient>&&, Ref<CookieJar>&&, UniqueRef<ProgressTrackerClient>&&);
+    WEBCORE_EXPORT PageConfiguration(PAL::SessionID, UniqueRef<EditorClient>&&, Ref<SocketProvider>&&, UniqueRef<LibWebRTCProvider>&&, Ref<CacheStorageProvider>&&, Ref<BackForwardClient>&&, Ref<CookieJar>&&, UniqueRef<ProgressTrackerClient>&&, UniqueRef<MediaRecorderProvider>&&);
     WEBCORE_EXPORT ~PageConfiguration();
     PageConfiguration(PageConfiguration&&);
 
@@ -123,6 +124,7 @@ public:
     RefPtr<DeviceOrientationUpdateProvider> deviceOrientationUpdateProvider;
 #endif
     Vector<String> corsDisablingPatterns;
+    UniqueRef<MediaRecorderProvider> mediaRecorderProvider;
 };
 
 }
index 10e91f3..ad51807 100644 (file)
@@ -26,6 +26,7 @@
 
 #include <wtf/CompletionHandler.h>
 #include <wtf/Forward.h>
+#include "Exception.h"
 
 #if ENABLE(MEDIA_STREAM)
 
@@ -43,12 +44,19 @@ class SharedBuffer;
 
 class MediaRecorderPrivate {
 public:
-    virtual void sampleBufferUpdated(MediaStreamTrackPrivate&, MediaSample&) = 0;
-    virtual void audioSamplesAvailable(MediaStreamTrackPrivate&, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) = 0;
+    virtual ~MediaRecorderPrivate() = default;
+
+    virtual void sampleBufferUpdated(const MediaStreamTrackPrivate&, MediaSample&) = 0;
+    virtual void audioSamplesAvailable(const MediaStreamTrackPrivate&, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) = 0;
 
     virtual void fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&, const String& mimeType)>&&) = 0;
-    virtual ~MediaRecorderPrivate() = default;
-    virtual void stopRecording() { }
+    virtual void stopRecording() { };
+
+    using ErrorCallback = Function<void(Optional<Exception>&&)>;
+    void setErrorCallback(ErrorCallback&& errorCallback) { m_errorCallback = WTFMove(errorCallback); }
+
+protected:
+    ErrorCallback m_errorCallback;
 };
     
 } // namespace WebCore
index 9183046..6a24a20 100644 (file)
@@ -40,7 +40,6 @@ std::unique_ptr<MediaRecorderPrivateAVFImpl> MediaRecorderPrivateAVFImpl::create
 {
     // FIXME: we will need to implement support for multiple audio/video tracks
     // Currently we only choose the first track as the recorded track.
-    // FIXME: We would better to throw an exception to JavaScript if writer creation fails.
 
     String audioTrackId;
     String videoTrackId;
@@ -82,14 +81,14 @@ MediaRecorderPrivateAVFImpl::MediaRecorderPrivateAVFImpl(Ref<MediaRecorderPrivat
 {
 }
 
-void MediaRecorderPrivateAVFImpl::sampleBufferUpdated(MediaStreamTrackPrivate& track, MediaSample& sampleBuffer)
+void MediaRecorderPrivateAVFImpl::sampleBufferUpdated(const MediaStreamTrackPrivate& track, MediaSample& sampleBuffer)
 {
     if (track.id() != m_recordedVideoTrackID)
         return;
     m_writer->appendVideoSampleBuffer(sampleBuffer.platformSample().sample.cmSampleBuffer);
 }
 
-void MediaRecorderPrivateAVFImpl::audioSamplesAvailable(MediaStreamTrackPrivate& track, const WTF::MediaTime& mediaTime, const PlatformAudioData& data, const AudioStreamDescription& description, size_t sampleCount)
+void MediaRecorderPrivateAVFImpl::audioSamplesAvailable(const MediaStreamTrackPrivate& track, const WTF::MediaTime& mediaTime, const PlatformAudioData& data, const AudioStreamDescription& description, size_t sampleCount)
 {
     if (track.id() != m_recordedAudioTrackID)
         return;
index 5ec0bee..0cf0df3 100644 (file)
@@ -43,8 +43,8 @@ private:
 
     friend std::unique_ptr<MediaRecorderPrivateAVFImpl> std::make_unique<MediaRecorderPrivateAVFImpl>(Ref<MediaRecorderPrivateWriter>&&, String&&, String&&);
 
-    void sampleBufferUpdated(MediaStreamTrackPrivate&, MediaSample&) final;
-    void audioSamplesAvailable(MediaStreamTrackPrivate&, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final;
+    void sampleBufferUpdated(const MediaStreamTrackPrivate&, MediaSample&) final;
+    void audioSamplesAvailable(const MediaStreamTrackPrivate&, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final;
     void fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&, const String&)>&&) final;
     const String& mimeType();
     void stopRecording();
index c816e1a..f2ac8c2 100644 (file)
 
 namespace WebCore {
 
-void MediaRecorderPrivateMock::sampleBufferUpdated(MediaStreamTrackPrivate& track, MediaSample&)
+void MediaRecorderPrivateMock::sampleBufferUpdated(const MediaStreamTrackPrivate& track, MediaSample&)
 {
     generateMockString(track);
 }
 
-void MediaRecorderPrivateMock::audioSamplesAvailable(MediaStreamTrackPrivate& track, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t)
+void MediaRecorderPrivateMock::audioSamplesAvailable(const MediaStreamTrackPrivate& track, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t)
 {
     generateMockString(track);
 }
 
-void MediaRecorderPrivateMock::generateMockString(MediaStreamTrackPrivate& track)
+void MediaRecorderPrivateMock::generateMockString(const MediaStreamTrackPrivate& track)
 {
     auto locker = holdLock(m_bufferLock);
     if (track.type() == RealtimeMediaSource::Type::Audio)
index 035711e..4daed67 100644 (file)
@@ -36,12 +36,12 @@ class MediaStreamTrackPrivate;
 
 class WEBCORE_EXPORT MediaRecorderPrivateMock final : public MediaRecorderPrivate {
 private:
-    void sampleBufferUpdated(MediaStreamTrackPrivate&, MediaSample&) final;
-    void audioSamplesAvailable(MediaStreamTrackPrivate&, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final;
+    void sampleBufferUpdated(const MediaStreamTrackPrivate&, MediaSample&) final;
+    void audioSamplesAvailable(const MediaStreamTrackPrivate&, const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final;
     void fetchData(CompletionHandler<void(RefPtr<SharedBuffer>&&, const String&)>&&) final;
     const String& mimeType();
     
-    void generateMockString(MediaStreamTrackPrivate&);
+    void generateMockString(const MediaStreamTrackPrivate&);
 
     mutable Lock m_bufferLock;
     StringBuilder m_buffer;
index fff45d6..51978bb 100644 (file)
@@ -49,9 +49,10 @@ class AudioStreamDescription;
 class MediaStreamTrackPrivate;
 class PlatformAudioData;
 
-class MediaRecorderPrivateWriter : public ThreadSafeRefCounted<MediaRecorderPrivateWriter, WTF::DestructionThread::Main>, public CanMakeWeakPtr<MediaRecorderPrivateWriter> {
+class WEBCORE_EXPORT MediaRecorderPrivateWriter : public ThreadSafeRefCounted<MediaRecorderPrivateWriter, WTF::DestructionThread::Main>, public CanMakeWeakPtr<MediaRecorderPrivateWriter> {
 public:
     static RefPtr<MediaRecorderPrivateWriter> create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack);
+    static RefPtr<MediaRecorderPrivateWriter> create(bool hasAudio, int width, int height);
     ~MediaRecorderPrivateWriter();
     
     bool setupWriter();
index 92fe2a4..069c887 100644 (file)
@@ -91,6 +91,17 @@ static NSString *getAVEncoderBitRateKeyWithFallback()
 
 RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(const MediaStreamTrackPrivate* audioTrack, const MediaStreamTrackPrivate* videoTrack)
 {
+    int width = 0, height = 0;
+    if (videoTrack) {
+        auto& settings = videoTrack->settings();
+        width = settings.width();
+        height = settings.height();
+    }
+    return create(!!audioTrack, width, height);
+}
+
+RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(bool hasAudio, int width, int height)
+{
     NSString *directory = FileSystem::createTemporaryDirectory(@"videos");
     NSString *filename = [NSString stringWithFormat:@"/%lld.mp4", CMClockGetTime(CMClockGetHostTimeClock()).value];
     NSString *path = [directory stringByAppendingString:filename];
@@ -106,12 +117,11 @@ RefPtr<MediaRecorderPrivateWriter> MediaRecorderPrivateWriter::create(const Medi
 
     auto writer = adoptRef(*new MediaRecorderPrivateWriter(WTFMove(avAssetWriter), WTFMove(filePath)));
 
-    if (audioTrack && !writer->setAudioInput())
+    if (hasAudio && !writer->setAudioInput())
         return nullptr;
 
-    if (videoTrack) {
-        auto& settings = videoTrack->settings();
-        if (!writer->setVideoInput(settings.width(), settings.height()))
+    if (width && height) {
+        if (!writer->setVideoInput(width, height))
             return nullptr;
     }
 
index 4a06844..de44d88 100644 (file)
@@ -81,7 +81,7 @@ public:
 
     String id() const { return m_id; }
 
-    MediaStreamTrackPrivateVector tracks() const;
+    WEBCORE_EXPORT MediaStreamTrackPrivateVector tracks() const;
     MediaStreamTrackPrivate* activeVideoTrack() { return m_activeVideoTrack; }
 
     bool active() const { return m_isActive; }
index 391a653..00716fd 100644 (file)
@@ -100,7 +100,7 @@ public:
     Ref<MediaStreamTrackPrivate> clone();
 
     RealtimeMediaSource& source() { return m_source.get(); }
-    RealtimeMediaSource::Type type() const;
+    WEBCORE_EXPORT RealtimeMediaSource::Type type() const;
 
     void endTrack();
 
@@ -110,7 +110,7 @@ public:
     bool hasObserver(Observer&) const;
 #endif
 
-    const RealtimeMediaSourceSettings& settings() const;
+    WEBCORE_EXPORT const RealtimeMediaSourceSettings& settings() const;
     const RealtimeMediaSourceCapabilities& capabilities() const;
 
     void applyConstraints(const MediaConstraints&, RealtimeMediaSource::ApplyConstraintsHandler&&);
index ba9418c..68ab423 100644 (file)
 #include "MediaEngineConfigurationFactory.h"
 #include "MediaPlayer.h"
 #include "MediaProducer.h"
+#include "MediaRecorderProvider.h"
 #include "MediaResourceLoader.h"
 #include "MediaStreamTrack.h"
 #include "MemoryCache.h"
@@ -550,6 +551,8 @@ void Internals::resetToConsistentState(Page& page)
 
 #if ENABLE(MEDIA_STREAM)
     RuntimeEnabledFeatures::sharedFeatures().setInterruptAudioOnPageVisibilityChangeEnabled(false);
+    WebCore::MediaRecorder::setCustomPrivateRecorderCreator(nullptr);
+    page.mediaRecorderProvider().setUseGPUProcess(true);
 #endif
 
     HTMLCanvasElement::setMaxPixelMemoryForTesting(0); // This means use the default value.
@@ -1558,7 +1561,9 @@ void Internals::setUseGPUProcessForWebRTC(bool useGPUProcess)
     auto* document = contextDocument();
     if (!document || !document->page())
         return;
+
     document->page()->libWebRTCProvider().setUseGPUProcess(useGPUProcess);
+    document->page()->mediaRecorderProvider().setUseGPUProcess(useGPUProcess);
 #endif
 }
 #endif
index ff52d25..532678f 100644 (file)
@@ -78,6 +78,8 @@ set(WebKit_INCLUDE_DIRECTORIES
     "${WEBKIT_DIR}/WebProcess/Gamepad"
     "${WEBKIT_DIR}/WebProcess/Geolocation"
     "${WEBKIT_DIR}/WebProcess/GPU"
+    "${WEBKIT_DIR}/WebProcess/GPU/media"
+    "${WEBKIT_DIR}/WebProcess/GPU/webrtc"
     "${WEBKIT_DIR}/WebProcess/IconDatabase"
     "${WEBKIT_DIR}/WebProcess/InjectedBundle"
     "${WEBKIT_DIR}/WebProcess/InjectedBundle/API"
index 4d5d587..a1dfd7a 100644 (file)
@@ -1,5 +1,72 @@
 2020-01-07  youenn fablet  <youenn@apple.com>
 
+        Implement MediaRecorder backend in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=205802
+
+        Reviewed by Eric Carlson.
+
+        Add support for sending audio/video tracks to record from WebProcess to GPUProcess.
+        Add a MediaRecorderPrivate implementation that supports sending one audio track and/or one video track to GPUProcess
+        and stopping/fetching data from the remote recorder in GPUProcess.
+
+        In GPUProcess, implement the remote recorder using the existing WebCore recorder writer.
+
+        * CMakeLists.txt:
+        * DerivedSources-input.xcfilelist:
+        * DerivedSources-output.xcfilelist:
+        * DerivedSources.make:
+        * GPUProcess/GPUConnectionToWebProcess.cpp:
+        (WebKit::GPUConnectionToWebProcess::userMediaCaptureManagerProxy):
+        (WebKit::GPUConnectionToWebProcess::mediaRecorderManager):
+        (WebKit::GPUConnectionToWebProcess::didReceiveMessage):
+        * GPUProcess/GPUConnectionToWebProcess.h:
+        * GPUProcess/mac/com.apple.WebKit.GPUProcess.sb.in:
+        * GPUProcess/webrtc/RemoteMediaRecorder.cpp: Added.
+        (WebKit::RemoteMediaRecorder::create):
+        (WebKit::RemoteMediaRecorder::RemoteMediaRecorder):
+        (WebKit::RemoteMediaRecorder::~RemoteMediaRecorder):
+        (WebKit::RemoteMediaRecorder::storage):
+        (WebKit::RemoteMediaRecorder::audioSamplesStorageChanged):
+        (WebKit::RemoteMediaRecorder::audioSamplesAvailable):
+        (WebKit::RemoteMediaRecorder::videoSampleAvailable):
+        (WebKit::RemoteMediaRecorder::fetchData):
+        (WebKit::RemoteMediaRecorder::stopRecording):
+        * GPUProcess/webrtc/RemoteMediaRecorder.h: Added.
+        * GPUProcess/webrtc/RemoteMediaRecorder.messages.in: Added.
+        * GPUProcess/webrtc/RemoteMediaRecorderManager.cpp: Added.
+        (WebKit::RemoteMediaRecorderManager::RemoteMediaRecorderManager):
+        (WebKit::RemoteMediaRecorderManager::~RemoteMediaRecorderManager):
+        (WebKit::RemoteMediaRecorderManager::didReceiveRemoteMediaRecorderMessage):
+        (WebKit::RemoteMediaRecorderManager::createRecorder):
+        (WebKit::RemoteMediaRecorderManager::releaseRecorder):
+        * GPUProcess/webrtc/RemoteMediaRecorderManager.h: Added.
+        (WebKit::RemoteMediaRecorderManager::didReceiveMessageFromWebProcess):
+        * GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in: Added.
+        * Scripts/webkit/messages.py:
+        * Sources.txt:
+        * SourcesCocoa.txt:
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/GPU/media/MediaRecorderProvider.cpp: Added.
+        (WebCore::MediaRecorderProvider::createMediaRecorderPrivate):
+        * WebProcess/GPU/media/MediaRecorderProvider.h: Added.
+        * WebProcess/GPU/webrtc/MediaRecorderIdentifier.h: Added.
+        * WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp: Added.
+        (WebKit::MediaRecorderPrivate::MediaRecorderPrivate):
+        (WebKit::MediaRecorderPrivate::~MediaRecorderPrivate):
+        (WebKit::MediaRecorderPrivate::sampleBufferUpdated):
+        (WebKit::MediaRecorderPrivate::audioSamplesAvailable):
+        (WebKit::MediaRecorderPrivate::storageChanged):
+        (WebKit::MediaRecorderPrivate::fetchData):
+        (WebKit::MediaRecorderPrivate::stopRecording):
+        * WebProcess/GPU/webrtc/MediaRecorderPrivate.h: Added.
+        * WebProcess/GPU/webrtc/MediaRecorderProvider.cpp: Added.
+        (WebKit::MediaRecorderProvider::createMediaRecorderPrivate):
+        * WebProcess/GPU/webrtc/MediaRecorderProvider.h: Added.
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::m_overriddenMediaType):
+
+2020-01-07  youenn fablet  <youenn@apple.com>
+
         Add an option to make video capture in GPUProcess
         https://bugs.webkit.org/show_bug.cgi?id=205853
 
index 7994e3c..edfb6e1 100644 (file)
@@ -19,6 +19,8 @@ $(PROJECT_DIR)/GPUProcess/mac/com.apple.WebKit.GPUProcess.sb.in
 $(PROJECT_DIR)/GPUProcess/media/RemoteMediaPlayerManagerProxy.messages.in
 $(PROJECT_DIR)/GPUProcess/media/RemoteMediaResourceManager.messages.in
 $(PROJECT_DIR)/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
+$(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
+$(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in
 $(PROJECT_DIR)/NetworkProcess/Cookies/WebCookieManager.messages.in
 $(PROJECT_DIR)/NetworkProcess/CustomProtocols/LegacyCustomProtocolManager.messages.in
 $(PROJECT_DIR)/NetworkProcess/IndexedDB/WebIDBServer.messages.in
index 0cd23e0..dfbc0ef 100644 (file)
@@ -129,6 +129,13 @@ $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaPlayerManagerMessagesRep
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaPlayerManagerProxyMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaPlayerManagerProxyMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaPlayerManagerProxyMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaRecorderManagerMessageReceiver.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaRecorderManagerMessages.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaRecorderManagerMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaRecorderMessageReceiver.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaRecorderMessages.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaRecorderMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaRecordersMessagesReplies.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaResourceManagerMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaResourceManagerMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteMediaResourceManagerMessagesReplies.h
index 9d3e2a9..e290930 100644 (file)
@@ -143,6 +143,8 @@ MESSAGE_RECEIVERS = \
     RemoteLayerTreeDrawingAreaProxy \
     RemoteMediaPlayerManager \
     RemoteMediaPlayerManagerProxy \
+    RemoteMediaRecorder \
+    RemoteMediaRecorderManager \
     RemoteMediaResourceManager \
     RemoteObjectRegistry \
     RemoteScrollingCoordinator \
index 52a95ce..620b41f 100644 (file)
@@ -39,6 +39,9 @@
 #include "RemoteLayerTreeDrawingAreaProxyMessages.h"
 #include "RemoteMediaPlayerManagerProxy.h"
 #include "RemoteMediaPlayerManagerProxyMessages.h"
+#include "RemoteMediaRecorderManager.h"
+#include "RemoteMediaRecorderManagerMessages.h"
+#include "RemoteMediaRecorderMessages.h"
 #include "RemoteMediaResourceManager.h"
 #include "RemoteMediaResourceManagerMessages.h"
 #include "RemoteScrollingCoordinatorTransaction.h"
@@ -137,6 +140,14 @@ UserMediaCaptureManagerProxy& GPUConnectionToWebProcess::userMediaCaptureManager
 
     return *m_userMediaCaptureManagerProxy;
 }
+
+RemoteMediaRecorderManager& GPUConnectionToWebProcess::mediaRecorderManager()
+{
+    if (!m_remoteMediaRecorderManager)
+        m_remoteMediaRecorderManager = makeUnique<RemoteMediaRecorderManager>(*this);
+
+    return *m_remoteMediaRecorderManager;
+}
 #endif
 
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
@@ -163,6 +174,14 @@ void GPUConnectionToWebProcess::didReceiveMessage(IPC::Connection& connection, I
         userMediaCaptureManagerProxy().didReceiveMessageFromGPUProcess(connection, decoder);
         return;
     }
+    if (decoder.messageReceiverName() == Messages::RemoteMediaRecorderManager::messageReceiverName()) {
+        mediaRecorderManager().didReceiveMessageFromWebProcess(connection, decoder);
+        return;
+    }
+    if (decoder.messageReceiverName() == Messages::RemoteMediaRecorder::messageReceiverName()) {
+        mediaRecorderManager().didReceiveRemoteMediaRecorderMessage(connection, decoder);
+        return;
+    }
 #endif
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
     if (decoder.messageReceiverName() == Messages::LibWebRTCCodecsProxy::messageReceiverName()) {
index ac7125d..f485964 100644 (file)
@@ -41,8 +41,9 @@ namespace WebKit {
 class GPUProcess;
 class LibWebRTCCodecsProxy;
 class RemoteMediaPlayerManagerProxy;
-class UserMediaCaptureManagerProxy;
+class RemoteMediaRecorderManager;
 class RemoteMediaResourceManager;
+class UserMediaCaptureManagerProxy;
 
 class GPUConnectionToWebProcess
     : public RefCounted<GPUConnectionToWebProcess>
@@ -71,6 +72,7 @@ private:
 #endif
 #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
     UserMediaCaptureManagerProxy& userMediaCaptureManagerProxy();
+    RemoteMediaRecorderManager& mediaRecorderManager();
 #endif
 
     // IPC::Connection::Client
@@ -89,6 +91,7 @@ private:
     PAL::SessionID m_sessionID;
 #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
     std::unique_ptr<UserMediaCaptureManagerProxy> m_userMediaCaptureManagerProxy;
+    std::unique_ptr<RemoteMediaRecorderManager> m_remoteMediaRecorderManager;
 #endif
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
     std::unique_ptr<LibWebRTCCodecsProxy> m_libWebRTCCodecsProxy;
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp
new file mode 100644 (file)
index 0000000..ad8a85d
--- /dev/null
@@ -0,0 +1,139 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "RemoteMediaRecorder.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
+
+#include "SharedRingBufferStorage.h"
+#include <WebCore/CARingBuffer.h>
+#include <WebCore/ImageTransferSessionVT.h>
+#include <WebCore/RemoteVideoSample.h>
+#include <WebCore/WebAudioBufferList.h>
+#include <wtf/CompletionHandler.h>
+
+using namespace WebCore;
+
+namespace WebKit {
+
+std::unique_ptr<RemoteMediaRecorder> RemoteMediaRecorder::create(GPUConnectionToWebProcess& gpuConnectionToWebProcess, MediaRecorderIdentifier identifier, bool recordAudio, int width, int height)
+{
+    auto writer = MediaRecorderPrivateWriter::create(recordAudio, width, height);
+    if (!writer)
+        return nullptr;
+    return std::unique_ptr<RemoteMediaRecorder>(new RemoteMediaRecorder { gpuConnectionToWebProcess, identifier, writer.releaseNonNull(), recordAudio });
+}
+
+RemoteMediaRecorder::RemoteMediaRecorder(GPUConnectionToWebProcess& gpuConnectionToWebProcess, MediaRecorderIdentifier identifier, Ref<MediaRecorderPrivateWriter>&& writer, bool recordAudio)
+    : m_gpuConnectionToWebProcess(gpuConnectionToWebProcess)
+    , m_identifier(identifier)
+    , m_writer(WTFMove(writer))
+{
+    if (recordAudio)
+        m_ringBuffer = makeUnique<CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(nullptr));
+}
+
+RemoteMediaRecorder::~RemoteMediaRecorder()
+{
+}
+
+SharedRingBufferStorage& RemoteMediaRecorder::storage()
+{
+    return static_cast<SharedRingBufferStorage&>(m_ringBuffer->storage());
+}
+
+void RemoteMediaRecorder::audioSamplesStorageChanged(const SharedMemory::Handle& handle, const WebCore::CAAudioStreamDescription& description, uint64_t numberOfFrames)
+{
+    ASSERT(m_ringBuffer);
+    if (!m_ringBuffer)
+        return;
+
+    m_description = description;
+
+    if (handle.isNull()) {
+        m_ringBuffer->deallocate();
+        storage().setReadOnly(false);
+        storage().setStorage(nullptr);
+        return;
+    }
+
+    auto memory = SharedMemory::map(handle, SharedMemory::Protection::ReadOnly);
+    storage().setStorage(WTFMove(memory));
+    storage().setReadOnly(true);
+
+    m_ringBuffer->allocate(description, numberOfFrames);
+}
+
+void RemoteMediaRecorder::audioSamplesAvailable(MediaTime time, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame)
+{
+    ASSERT(m_ringBuffer);
+    if (!m_ringBuffer)
+        return;
+
+    m_ringBuffer->setCurrentFrameBounds(startFrame, endFrame);
+
+    WebAudioBufferList audioData(m_description, numberOfFrames);
+    m_ringBuffer->fetch(audioData.list(), numberOfFrames, time.timeValue());
+
+    m_writer->appendAudioSampleBuffer(audioData, m_description, time, numberOfFrames);
+}
+
+void RemoteMediaRecorder::videoSampleAvailable(WebCore::RemoteVideoSample&& remoteSample)
+{
+    if (!m_imageTransferSession || m_imageTransferSession->pixelFormat() != remoteSample.videoFormat())
+        m_imageTransferSession = ImageTransferSessionVT::create(remoteSample.videoFormat());
+
+    if (!m_imageTransferSession) {
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    auto sampleBuffer = m_imageTransferSession->createMediaSample(remoteSample.surface(), remoteSample.time(), remoteSample.size());
+    if (!sampleBuffer) {
+        ASSERT_NOT_REACHED();
+        return;
+    }
+
+    m_writer->appendVideoSampleBuffer(sampleBuffer->platformSample().sample.cmSampleBuffer);
+}
+
+void RemoteMediaRecorder::fetchData(CompletionHandler<void(IPC::DataReference&&, const String& mimeType)>&& completionHandler)
+{
+    static NeverDestroyed<const String> mp4MimeType(MAKE_STATIC_STRING_IMPL("video/mp4"));
+    m_writer->fetchData([&mp4MimeType = mp4MimeType.get(), completionHandler = WTFMove(completionHandler)](auto&& data) mutable {
+        auto* pointer = reinterpret_cast<const uint8_t*>(data ? data->data() : nullptr);
+        completionHandler(IPC::DataReference { pointer, data ? data->size() : 0 }, mp4MimeType);
+    });
+}
+
+void RemoteMediaRecorder::stopRecording()
+{
+    m_writer->stopRecording();
+}
+
+}
+
+#endif
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h
new file mode 100644 (file)
index 0000000..44a8d17
--- /dev/null
@@ -0,0 +1,85 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
+
+#include "MediaRecorderIdentifier.h"
+#include "MessageReceiver.h"
+#include "SharedMemory.h"
+#include <WebCore/CAAudioStreamDescription.h>
+#include <WebCore/MediaRecorderPrivateWriterCocoa.h>
+#include <wtf/MediaTime.h>
+
+namespace IPC {
+class Connection;
+class DataReference;
+class Decoder;
+}
+
+namespace WebCore {
+class CARingBuffer;
+class ImageTransferSessionVT;
+class RemoteVideoSample;
+}
+
+namespace WebKit {
+
+class GPUConnectionToWebProcess;
+class SharedRingBufferStorage;
+
+class RemoteMediaRecorder : private IPC::MessageReceiver {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    static std::unique_ptr<RemoteMediaRecorder> create(GPUConnectionToWebProcess&, MediaRecorderIdentifier, bool recordAudio, int width, int height);
+    ~RemoteMediaRecorder();
+
+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
+
+private:
+    RemoteMediaRecorder(GPUConnectionToWebProcess&, MediaRecorderIdentifier, Ref<WebCore::MediaRecorderPrivateWriter>&&, bool recordAudio);
+
+    // IPC::MessageReceiver
+    void audioSamplesStorageChanged(const SharedMemory::Handle&, const WebCore::CAAudioStreamDescription&, uint64_t numberOfFrames);
+    void audioSamplesAvailable(MediaTime, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame);
+    void videoSampleAvailable(WebCore::RemoteVideoSample&&);
+    void fetchData(CompletionHandler<void(IPC::DataReference&&, const String& mimeType)>&&);
+    void stopRecording();
+
+    SharedRingBufferStorage& storage();
+
+    GPUConnectionToWebProcess& m_gpuConnectionToWebProcess;
+    MediaRecorderIdentifier m_identifier;
+    Ref<WebCore::MediaRecorderPrivateWriter> m_writer;
+
+    WebCore::CAAudioStreamDescription m_description;
+    std::unique_ptr<WebCore::CARingBuffer> m_ringBuffer;
+    std::unique_ptr<WebCore::ImageTransferSessionVT> m_imageTransferSession;
+};
+
+}
+
+#endif
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
new file mode 100644 (file)
index 0000000..54ba64d
--- /dev/null
@@ -0,0 +1,34 @@
+# Copyright (C) 2019 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(MEDIA_STREAM)
+
+messages -> RemoteMediaRecorder NotRefCounted {
+    AudioSamplesStorageChanged(WebKit::SharedMemory::Handle storageHandle, WebCore::CAAudioStreamDescription description, uint64_t numberOfFrames)
+    AudioSamplesAvailable(MediaTime time, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame)
+    VideoSampleAvailable(WebCore::RemoteVideoSample sample)
+    FetchData() -> (IPC::DataReference buffer, String mimeType) Async
+    StopRecording()
+}
+
+#endif
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.cpp
new file mode 100644 (file)
index 0000000..34e3fed
--- /dev/null
@@ -0,0 +1,72 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "RemoteMediaRecorderManager.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
+
+#include "DataReference.h"
+#include "Decoder.h"
+#include "RemoteMediaRecorder.h"
+#include <WebCore/ExceptionData.h>
+
+namespace WebKit {
+
+RemoteMediaRecorderManager::RemoteMediaRecorderManager(GPUConnectionToWebProcess& gpuConnectionToWebProcess)
+    : m_gpuConnectionToWebProcess(gpuConnectionToWebProcess)
+{
+}
+
+RemoteMediaRecorderManager::~RemoteMediaRecorderManager()
+{
+}
+
+void RemoteMediaRecorderManager::didReceiveRemoteMediaRecorderMessage(IPC::Connection& connection, IPC::Decoder& decoder)
+{
+    if (auto* recorder = m_recorders.get(makeObjectIdentifier<MediaRecorderIdentifierType>(decoder.destinationID())))
+        recorder->didReceiveMessage(connection, decoder);
+}
+
+void RemoteMediaRecorderManager::createRecorder(MediaRecorderIdentifier identifier, bool recordAudio, int width, int height, CompletionHandler<void(Optional<ExceptionData>&&)>&& completionHandler)
+{
+    ASSERT(!m_recorders.contains(identifier));
+    auto recorder = RemoteMediaRecorder::create(m_gpuConnectionToWebProcess, identifier, recordAudio, width, height);
+    if (!recorder)
+        return completionHandler(ExceptionData { NotSupportedError, "Unable to create a recorder with the provided stream"_s });
+
+    m_recorders.add(identifier, WTFMove(recorder));
+    completionHandler({ });
+}
+
+void RemoteMediaRecorderManager::releaseRecorder(MediaRecorderIdentifier identifier)
+{
+    ASSERT(m_recorders.contains(identifier));
+    m_recorders.remove(identifier);
+}
+
+}
+
+#endif
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.h b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.h
new file mode 100644 (file)
index 0000000..5d570b4
--- /dev/null
@@ -0,0 +1,72 @@
+
+
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
+
+#include "MediaRecorderIdentifier.h"
+#include "MessageReceiver.h"
+#include <wtf/Forward.h>
+#include <wtf/HashMap.h>
+
+namespace IPC {
+class Connection;
+class Decoder;
+}
+
+namespace WebCore {
+struct ExceptionData;
+}
+
+namespace WebKit {
+
+class GPUConnectionToWebProcess;
+class RemoteMediaRecorder;
+
+class RemoteMediaRecorderManager : private IPC::MessageReceiver {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    explicit RemoteMediaRecorderManager(GPUConnectionToWebProcess&);
+    ~RemoteMediaRecorderManager();
+
+    void didReceiveRemoteMediaRecorderMessage(IPC::Connection&, IPC::Decoder&);
+    void didReceiveMessageFromWebProcess(IPC::Connection& connection, IPC::Decoder& decoder) { didReceiveMessage(connection, decoder); }
+
+private:
+    // IPC::MessageReceiver
+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
+    void createRecorder(MediaRecorderIdentifier, bool recordAudio, int width, int height, CompletionHandler<void(Optional<WebCore::ExceptionData>&&)>&&);
+    void releaseRecorder(MediaRecorderIdentifier);
+
+    GPUConnectionToWebProcess& m_gpuConnectionToWebProcess;
+    HashMap<MediaRecorderIdentifier, std::unique_ptr<RemoteMediaRecorder>> m_recorders;
+};
+
+}
+
+#endif
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in b/Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in
new file mode 100644 (file)
index 0000000..387a413
--- /dev/null
@@ -0,0 +1,31 @@
+# Copyright (C) 2019 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(MEDIA_STREAM)
+
+messages -> RemoteMediaRecorderManager NotRefCounted {
+    CreateRecorder(WebKit::MediaRecorderIdentifier id, bool hasAudio, int width, int height) -> (Optional<WebCore::ExceptionData> creationError) Async
+    ReleaseRecorder(WebKit::MediaRecorderIdentifier id)
+}
+
+#endif
index 5d84471..f8bce14 100644 (file)
@@ -220,6 +220,7 @@ def types_that_cannot_be_forward_declared():
         'WebKit::ActivityStateChangeID',
         'WebKit::LayerHostingContextID',
         'WebKit::MediaPlayerPrivateRemoteIdentifier',
+        'WebKit::MediaRecorderIdentifier',
         'WebKit::RemoteMediaResourceIdentifier',
         'WebKit::RTCDecoderIdentifier',
         'WebKit::RTCEncoderIdentifier',
index d0393bb..306cd1e 100644 (file)
@@ -24,6 +24,8 @@
 GPUProcess/GPUProcess.cpp
 GPUProcess/GPUConnectionToWebProcess.cpp
 GPUProcess/GPUProcessCreationParameters.cpp
+GPUProcess/webrtc/RemoteMediaRecorder.cpp
+GPUProcess/webrtc/RemoteMediaRecorderManager.cpp
 GPUProcess/media/RemoteMediaPlayerManagerProxy.cpp
 GPUProcess/media/RemoteMediaPlayerProxy.cpp
 GPUProcess/media/RemoteMediaResource.cpp
@@ -513,6 +515,8 @@ WebProcess/GPU/media/RemoteMediaPlayerManager.cpp
 WebProcess/GPU/media/RemoteMediaPlayerMIMETypeCache.cpp @no-unify
 WebProcess/GPU/media/RemoteMediaResourceProxy.cpp
 WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
+WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
+WebProcess/GPU/webrtc/MediaRecorderProvider.cpp
 
 WebProcess/Network/NetworkProcessConnection.cpp
 WebProcess/Network/WebLoaderStrategy.cpp
index fe75795..abc8b85 100644 (file)
@@ -635,5 +635,7 @@ GPUProcessProxyMessageReceiver.cpp
 GPUProcessMessageReceiver.cpp
 LibWebRTCCodecsProxyMessageReceiver.cpp
 LibWebRTCCodecsMessageReceiver.cpp
+RemoteMediaRecorderMessageReceiver.cpp
+RemoteMediaRecorderManagerMessageReceiver.cpp
 ServiceWorkerFetchTaskMessageReceiver.cpp
 TextCheckingControllerProxyMessageReceiver.cpp
index 5778ba5..cf0b3be 100644 (file)
                4172198C23B612E800AE5686 /* RTCDecoderIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RTCDecoderIdentifier.h; sourceTree = "<group>"; };
                4172198D23B62C7C00AE5686 /* LibWebRTCCodecs.messages.in */ = {isa = PBXFileReference; lastKnownFileType = text; path = LibWebRTCCodecs.messages.in; sourceTree = "<group>"; };
                4176901322FDD41B00B1576D /* NetworkRTCProvider.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = NetworkRTCProvider.mm; sourceTree = "<group>"; };
+               4176E89223C34D6E003E83FE /* MediaRecorderProvider.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = MediaRecorderProvider.h; sourceTree = "<group>"; };
+               4176E89323C34D6F003E83FE /* MediaRecorderProvider.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = MediaRecorderProvider.cpp; sourceTree = "<group>"; };
+               4176E89423C34E26003E83FE /* MediaRecorderPrivate.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MediaRecorderPrivate.cpp; sourceTree = "<group>"; };
+               4176E89523C34E27003E83FE /* MediaRecorderPrivate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaRecorderPrivate.h; sourceTree = "<group>"; };
+               4176E89723C361BD003E83FE /* MediaRecorderIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaRecorderIdentifier.h; sourceTree = "<group>"; };
+               4176E89823C36677003E83FE /* RemoteMediaRecorder.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteMediaRecorder.h; sourceTree = "<group>"; };
+               4176E89923C36D8E003E83FE /* RemoteMediaRecorderManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteMediaRecorderManager.h; sourceTree = "<group>"; };
+               4176E89A23C36F71003E83FE /* RemoteMediaRecorderManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteMediaRecorderManager.cpp; sourceTree = "<group>"; };
+               4176E89B23C37222003E83FE /* RemoteMediaRecorderManager.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RemoteMediaRecorderManager.messages.in; sourceTree = "<group>"; };
+               4176E89C23C37479003E83FE /* RemoteMediaRecorder.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RemoteMediaRecorder.messages.in; sourceTree = "<group>"; };
+               4176E89D23C376D5003E83FE /* RemoteMediaRecorder.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteMediaRecorder.cpp; sourceTree = "<group>"; };
                417915AC2256BB7400D6F97E /* WebSocketChannel.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = WebSocketChannel.cpp; path = Network/WebSocketChannel.cpp; sourceTree = "<group>"; };
                417915AD2256BB7400D6F97E /* WebSocketChannel.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = WebSocketChannel.h; path = Network/WebSocketChannel.h; sourceTree = "<group>"; };
                417915B02256C0D600D6F97E /* WebSocketChannelManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = WebSocketChannelManager.h; path = Network/WebSocketChannelManager.h; sourceTree = "<group>"; };
                                4172198A23B6128200AE5686 /* LibWebRTCCodecs.cpp */,
                                4172198923B6128200AE5686 /* LibWebRTCCodecs.h */,
                                4172198D23B62C7C00AE5686 /* LibWebRTCCodecs.messages.in */,
+                               4176E89723C361BD003E83FE /* MediaRecorderIdentifier.h */,
+                               4176E89423C34E26003E83FE /* MediaRecorderPrivate.cpp */,
+                               4176E89523C34E27003E83FE /* MediaRecorderPrivate.h */,
+                               4176E89323C34D6F003E83FE /* MediaRecorderProvider.cpp */,
+                               4176E89223C34D6E003E83FE /* MediaRecorderProvider.h */,
                                4172198C23B612E800AE5686 /* RTCDecoderIdentifier.h */,
                                4158649A23BE092400A0A61E /* RTCEncoderIdentifier.h */,
                        );
                                41E0A7C623B6397800561060 /* LibWebRTCCodecsProxy.h */,
                                41E0A7C723B6397900561060 /* LibWebRTCCodecsProxy.messages.in */,
                                41E0A7C823B6397900561060 /* LibWebRTCCodecsProxy.mm */,
+                               4176E89D23C376D5003E83FE /* RemoteMediaRecorder.cpp */,
+                               4176E89823C36677003E83FE /* RemoteMediaRecorder.h */,
+                               4176E89C23C37479003E83FE /* RemoteMediaRecorder.messages.in */,
+                               4176E89A23C36F71003E83FE /* RemoteMediaRecorderManager.cpp */,
+                               4176E89923C36D8E003E83FE /* RemoteMediaRecorderManager.h */,
+                               4176E89B23C37222003E83FE /* RemoteMediaRecorderManager.messages.in */,
                        );
                        path = webrtc;
                        sourceTree = "<group>";
diff --git a/Source/WebKit/WebProcess/GPU/media/MediaRecorderProvider.cpp b/Source/WebKit/WebProcess/GPU/media/MediaRecorderProvider.cpp
new file mode 100644 (file)
index 0000000..de2d66b
--- /dev/null
@@ -0,0 +1,45 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "MediaRecorderProvider.h"
+
+#if PLATFORM(COCOA)
+#include "MediaRecorderPrivateAVFImpl.h"
+#endif
+
+namespace WebCore {
+
+std::unique_ptr<MediaRecorderPrivate> MediaRecorderProvider::createMediaRecorderPrivate(MediaStreamPrivate& stream)
+{
+#if PLATFORM(COCOA)
+    return MediaRecorderPrivateAVFImpl::create(stream);
+#else
+    UNUSED_PARAM(stream);
+    return nullptr;
+#endif
+}
+
+}
diff --git a/Source/WebKit/WebProcess/GPU/media/MediaRecorderProvider.h b/Source/WebKit/WebProcess/GPU/media/MediaRecorderProvider.h
new file mode 100644 (file)
index 0000000..710e8d4
--- /dev/null
@@ -0,0 +1,42 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+namespace WebCore {
+
+class MediaRecorderPrivate;
+class MediaStreamPrivate;
+
+class WEBCORE_EXPORT MediaRecorderProvider {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    static UniqueRef<MediaRecorderProvider> create() { return makeUniqueRef<MediaRecorderProvider>(); }
+    virtual ~MediaRecorderProvider() = default;
+
+    virtual std::unique_ptr<MediaRecorderPrivate> createMediaRecorderPrivate(MediaStreamPrivate&);
+};
+
+}
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderIdentifier.h b/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderIdentifier.h
new file mode 100644 (file)
index 0000000..32f04e3
--- /dev/null
@@ -0,0 +1,35 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include <wtf/ObjectIdentifier.h>
+
+namespace WebKit {
+
+enum MediaRecorderIdentifierType { };
+using MediaRecorderIdentifier = ObjectIdentifier<MediaRecorderIdentifierType>;
+
+} // namespace WebKit
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp b/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
new file mode 100644 (file)
index 0000000..08128d2
--- /dev/null
@@ -0,0 +1,146 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "MediaRecorderPrivate.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
+
+#include "GPUProcessConnection.h"
+#include "RemoteMediaRecorderManagerMessages.h"
+#include "RemoteMediaRecorderMessages.h"
+#include "WebProcess.h"
+#include <WebCore/CARingBuffer.h>
+#include <WebCore/MediaStreamPrivate.h>
+#include <WebCore/MediaStreamTrackPrivate.h>
+#include <WebCore/RemoteVideoSample.h>
+#include <WebCore/SharedBuffer.h>
+#include <WebCore/WebAudioBufferList.h>
+
+using namespace WebCore;
+
+namespace WebKit {
+
+MediaRecorderPrivate::MediaRecorderPrivate(const MediaStreamPrivate& stream)
+    : m_identifier(MediaRecorderIdentifier::generate())
+    , m_connection(WebProcess::singleton().ensureGPUProcessConnection().connection())
+{
+    // FIXME: we will need to implement support for multiple audio/video tracks
+    // Currently we only choose the first track as the recorded track.
+    // FIXME: We would better to throw an exception to JavaScript if writer creation fails.
+
+    const MediaStreamTrackPrivate* audioTrack { nullptr };
+    const MediaStreamTrackPrivate* videoTrack { nullptr };
+    for (auto& track : stream.tracks()) {
+        if (!track->enabled() || track->ended())
+            continue;
+        switch (track->type()) {
+        case RealtimeMediaSource::Type::Video: {
+            auto& settings = track->settings();
+            if (!videoTrack && settings.supportsWidth() && settings.supportsHeight()) {
+                videoTrack = track.get();
+                m_recordedVideoTrackID = videoTrack->id();
+            }
+            break;
+        }
+        case RealtimeMediaSource::Type::Audio:
+            if (!audioTrack) {
+                m_ringBuffer = makeUnique<CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(this));
+                audioTrack = track.get();
+                m_recordedAudioTrackID = audioTrack->id();
+            }
+            break;
+        case RealtimeMediaSource::Type::None:
+            break;
+        }
+    }
+    m_connection->sendWithAsyncReply(Messages::RemoteMediaRecorderManager::CreateRecorder { m_identifier, !!audioTrack, videoTrack ? videoTrack->settings().width() : 0, videoTrack ? videoTrack->settings().height() : 0 }, [this, weakThis = makeWeakPtr(this)](auto&& exception) {
+        if (!weakThis || !exception)
+            return;
+        m_errorCallback(Exception { exception->code, WTFMove(exception->message) });
+    }, 0);
+}
+
+MediaRecorderPrivate::~MediaRecorderPrivate()
+{
+    m_connection->send(Messages::RemoteMediaRecorderManager::ReleaseRecorder { m_identifier }, 0);
+}
+
+void MediaRecorderPrivate::sampleBufferUpdated(const WebCore::MediaStreamTrackPrivate& track, WebCore::MediaSample& sample)
+{
+    if (track.id() != m_recordedVideoTrackID)
+        return;
+    if (auto remoteSample = RemoteVideoSample::create(sample))
+        m_connection->send(Messages::RemoteMediaRecorder::VideoSampleAvailable { WTFMove(*remoteSample) }, m_identifier);
+}
+
+void MediaRecorderPrivate::audioSamplesAvailable(const WebCore::MediaStreamTrackPrivate& track, const MediaTime& time, const PlatformAudioData& audioData, const AudioStreamDescription& description, size_t numberOfFrames)
+{
+    if (track.id() != m_recordedAudioTrackID)
+        return;
+
+    if (m_description != description) {
+        ASSERT(description.platformDescription().type == PlatformDescription::CAAudioStreamBasicType);
+        m_description = *WTF::get<const AudioStreamBasicDescription*>(description.platformDescription().description);
+
+        // Allocate a ring buffer large enough to contain 2 seconds of audio.
+        m_numberOfFrames = m_description.sampleRate() * 2;
+        m_ringBuffer->allocate(m_description.streamDescription(), m_numberOfFrames);
+    }
+
+    ASSERT(is<WebAudioBufferList>(audioData));
+    m_ringBuffer->store(downcast<WebAudioBufferList>(audioData).list(), numberOfFrames, time.timeValue());
+    uint64_t startFrame;
+    uint64_t endFrame;
+    m_ringBuffer->getCurrentFrameBounds(startFrame, endFrame);
+    m_connection->send(Messages::RemoteMediaRecorder::AudioSamplesAvailable { time, numberOfFrames, startFrame, endFrame }, m_identifier);
+}
+
+void MediaRecorderPrivate::storageChanged(SharedMemory* storage)
+{
+    SharedMemory::Handle handle;
+    if (storage)
+        storage->createHandle(handle, SharedMemory::Protection::ReadOnly);
+    m_connection->send(Messages::RemoteMediaRecorder::AudioSamplesStorageChanged { handle, m_description, static_cast<uint64_t>(m_numberOfFrames) }, m_identifier);
+}
+
+void MediaRecorderPrivate::fetchData(CompletionHandler<void(RefPtr<WebCore::SharedBuffer>&&, const String& mimeType)>&& completionHandler)
+{
+    m_connection->sendWithAsyncReply(Messages::RemoteMediaRecorder::FetchData { }, [completionHandler = WTFMove(completionHandler)](auto&& data, auto&& mimeType) mutable {
+        RefPtr<SharedBuffer> buffer;
+        if (data.size())
+            buffer = SharedBuffer::create(data.data(), data.size());
+        completionHandler(WTFMove(buffer), mimeType);
+    }, m_identifier);
+}
+
+void MediaRecorderPrivate::stopRecording()
+{
+    m_connection->send(Messages::RemoteMediaRecorder::StopRecording { }, m_identifier);
+}
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h b/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h
new file mode 100644 (file)
index 0000000..8d3e8bf
--- /dev/null
@@ -0,0 +1,76 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
+
+#include "MediaRecorderIdentifier.h"
+#include "SharedRingBufferStorage.h"
+#include <WebCore/MediaRecorderPrivate.h>
+#include <wtf/MediaTime.h>
+#include <wtf/WeakPtr.h>
+
+namespace IPC {
+class Connection;
+}
+
+namespace WebCore {
+class MediaStreamPrivate;
+}
+
+namespace WebKit {
+
+class MediaRecorderPrivate final : public WebCore::MediaRecorderPrivate, public SharedRingBufferStorage::Client, public CanMakeWeakPtr<MediaRecorderPrivate> {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    explicit MediaRecorderPrivate(const WebCore::MediaStreamPrivate&);
+    ~MediaRecorderPrivate();
+
+private:
+    // WebCore::MediaRecorderPrivate
+    void sampleBufferUpdated(const WebCore::MediaStreamTrackPrivate&, WebCore::MediaSample&) final;
+    void audioSamplesAvailable(const WebCore::MediaStreamTrackPrivate&, const WTF::MediaTime&, const WebCore::PlatformAudioData&, const WebCore::AudioStreamDescription&, size_t) final;
+    void fetchData(CompletionHandler<void(RefPtr<WebCore::SharedBuffer>&&, const String& mimeType)>&&) final;
+    void stopRecording() final;
+
+    // SharedRingBufferStorage::Client
+    void storageChanged(SharedMemory*);
+
+    MediaRecorderIdentifier m_identifier;
+
+    Ref<IPC::Connection> m_connection;
+    String m_recordedAudioTrackID;
+    String m_recordedVideoTrackID;
+
+    std::unique_ptr<WebCore::CARingBuffer> m_ringBuffer;
+    WebCore::CAAudioStreamDescription m_description { };
+    int64_t m_numberOfFrames { 0 };
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS)
+
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.cpp b/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.cpp
new file mode 100644 (file)
index 0000000..58b7f7e
--- /dev/null
@@ -0,0 +1,50 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "MediaRecorderProvider.h"
+
+#include "MediaRecorderPrivate.h"
+#include <WebCore/MediaRecorderPrivate.h>
+
+using namespace WebCore;
+
+namespace WebKit {
+
+#if ENABLE(MEDIA_STREAM)
+std::unique_ptr<WebCore::MediaRecorderPrivate> MediaRecorderProvider::createMediaRecorderPrivate(const MediaStreamPrivate& stream)
+{
+#if ENABLE(GPU_PROCESS)
+    if (!m_useGPUProcess)
+        return WebCore::MediaRecorderProvider::createMediaRecorderPrivate(stream);
+
+    return makeUnique<MediaRecorderPrivate>(stream);
+#else
+    return WebCore::MediaRecorderProvider::createMediaRecorderPrivate(stream);
+#endif // ENABLE(GPU_PROCESS)
+}
+#endif // ENABLE(MEDIA_STREAM)
+
+}
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.h b/Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.h
new file mode 100644 (file)
index 0000000..045faa4
--- /dev/null
@@ -0,0 +1,42 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include <WebCore/MediaRecorderProvider.h>
+
+namespace WebKit {
+
+class MediaRecorderProvider final : public WebCore::MediaRecorderProvider {
+public:
+    MediaRecorderProvider() = default;
+
+private:
+#if ENABLE(MEDIA_STREAM)
+    std::unique_ptr<WebCore::MediaRecorderPrivate> createMediaRecorderPrivate(const WebCore::MediaStreamPrivate&) final;
+#endif
+};
+
+}
index 1e3ef77..396870c 100644 (file)
@@ -45,6 +45,7 @@
 #include "LibWebRTCProvider.h"
 #include "LoadParameters.h"
 #include "Logging.h"
+#include "MediaRecorderProvider.h"
 #include "NetscapePlugin.h"
 #include "NetworkConnectionToWebProcessMessages.h"
 #include "NetworkProcessConnection.h"
@@ -461,7 +462,8 @@ WebPage::WebPage(PageIdentifier pageID, WebPageCreationParameters&& parameters)
         WebProcess::singleton().cacheStorageProvider(),
         WebBackForwardListProxy::create(*this),
         WebCookieJar::create(),
-        makeUniqueRef<WebProgressTrackerClient>(*this)
+        makeUniqueRef<WebProgressTrackerClient>(*this),
+        makeUniqueRef<MediaRecorderProvider>()
     );
     pageConfiguration.chromeClient = new WebChromeClient(*this);
 #if ENABLE(CONTEXT_MENUS)
index 4c44cde..d8893a8 100644 (file)
@@ -1,3 +1,14 @@
+2020-01-07  youenn fablet  <youenn@apple.com>
+
+        Implement MediaRecorder backend in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=205802
+
+        Reviewed by Eric Carlson.
+
+        * WebView/WebView.mm:
+        (-[WebView _commonInitializationWithFrameName:groupName:]):
+        (-[WebView initSimpleHTMLDocumentWithStyle:frame:preferences:groupName:]):
+
 2020-01-07  Carlos Garcia Campos  <cgarcia@igalia.com>
 
         [GTK][WPE] Add API to set purpose and hints of active editable element to input methods
index 4c41864..19b221c 100644 (file)
 #import <WebCore/LocalizedStrings.h>
 #import <WebCore/LogInitialization.h>
 #import <WebCore/MIMETypeRegistry.h>
+#import <WebCore/MediaRecorderProvider.h>
 #import <WebCore/MemoryCache.h>
 #import <WebCore/MemoryRelease.h>
 #import <WebCore/NetworkStorageSession.h>
@@ -1448,7 +1449,8 @@ static void WebKitInitializeGamepadProviderIfNecessary()
         WebCore::CacheStorageProvider::create(),
         BackForwardList::create(self),
         WebCore::CookieJar::create(storageProvider.copyRef()),
-        makeUniqueRef<WebProgressTrackerClient>(self)
+        makeUniqueRef<WebProgressTrackerClient>(self),
+        makeUniqueRef<WebCore::MediaRecorderProvider>()
     );
 #if !PLATFORM(IOS_FAMILY)
     pageConfiguration.chromeClient = new WebChromeClient(self);
@@ -1717,7 +1719,8 @@ static void WebKitInitializeGamepadProviderIfNecessary()
         WebCore::CacheStorageProvider::create(),
         BackForwardList::create(self),
         WebCore::CookieJar::create(storageProvider.copyRef()),
-        makeUniqueRef<WebProgressTrackerClient>(self)
+        makeUniqueRef<WebProgressTrackerClient>(self),
+        makeUniqueRef<WebCore::MediaRecorderProvider>()
     );
     pageConfiguration.chromeClient = new WebChromeClientIOS(self);
 #if ENABLE(DRAG_SUPPORT)
index 1955823..3c8bc2c 100644 (file)
@@ -1,3 +1,13 @@
+2020-01-07  youenn fablet  <youenn@apple.com>
+
+        Implement MediaRecorder backend in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=205802
+
+        Reviewed by Eric Carlson.
+
+        * WebView.cpp:
+        (WebView::initWithFrame):
+
 2020-01-07  Carlos Garcia Campos  <cgarcia@igalia.com>
 
         [GTK][WPE] Add API to set purpose and hints of active editable element to input methods
index eb08664..e9573d2 100644 (file)
 #include <WebCore/LogInitialization.h>
 #include <WebCore/Logging.h>
 #include <WebCore/MIMETypeRegistry.h>
+#include <WebCore/MediaRecorderProvider.h>
 #include <WebCore/MemoryCache.h>
 #include <WebCore/MemoryRelease.h>
 #include <WebCore/NetworkStorageSession.h>
@@ -3118,10 +3119,11 @@ HRESULT WebView::initWithFrame(RECT frame, _In_ BSTR frameName, _In_ BSTR groupN
         makeUniqueRef<WebEditorClient>(this),
         SocketProvider::create(),
         makeUniqueRef<LibWebRTCProvider>(),
-        WebCore::CacheStorageProvider::create(),
+        CacheStorageProvider::create(),
         BackForwardList::create(),
         CookieJar::create(storageProvider.copyRef()),
-        makeUniqueRef<WebProgressTrackerClient>()
+        makeUniqueRef<WebProgressTrackerClient>(),
+        makeUniqueRef<MediaRecorderProvider>()
     );
     configuration.chromeClient = new WebChromeClient(this);
     configuration.contextMenuClient = new WebContextMenuClient(this);