Add support for MediaStream audio track rendering in GPUProcess
authoryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 15 Jan 2020 11:30:52 +0000 (11:30 +0000)
committeryouenn@apple.com <youenn@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 15 Jan 2020 11:30:52 +0000 (11:30 +0000)
https://bugs.webkit.org/show_bug.cgi?id=206175

Reviewed by Eric Carlson.

Source/WebCore:

Simplify model to use start/stop instead of setPaused.
Simplify and fix issue in computation of the muted state of the renderer.
Covered by existing tests run with GPU process enabled and manual testing

* platform/mediastream/AudioMediaStreamTrackRenderer.h:
* platform/mediastream/AudioTrackPrivateMediaStream.cpp:
(WebCore::AudioTrackPrivateMediaStream::playInternal):
(WebCore::AudioTrackPrivateMediaStream::pause):
(WebCore::AudioTrackPrivateMediaStream::audioSamplesAvailable):
(WebCore::AudioTrackPrivateMediaStream::updateRendererMutedState):
* platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.cpp:
(WebCore::AudioMediaStreamTrackRendererCocoa::start):
(WebCore::AudioMediaStreamTrackRendererCocoa::stop):
(WebCore::AudioMediaStreamTrackRendererCocoa::clear):
(WebCore::AudioMediaStreamTrackRendererCocoa::pushSamples):
(WebCore::AudioMediaStreamTrackRendererCocoa::render):
* platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.h:

Source/WebKit:

Implement an AudioMediaStreamTrackRenderer at WebKit level by creating a remote renderer in GPUProcess and sending IPC to pass
audio data as well as orders (start/stop/setMuted).

Implement the remote renderer using WebCore audio track renderer.

Enable WebKit remote renderer F GPU process for media is enabled.

* DerivedSources-input.xcfilelist:
* DerivedSources-output.xcfilelist:
* DerivedSources.make:
* GPUProcess/GPUConnectionToWebProcess.cpp:
(WebKit::GPUConnectionToWebProcess::audioTrackRendererManager):
(WebKit::GPUConnectionToWebProcess::didReceiveMessage):
* GPUProcess/GPUConnectionToWebProcess.h:
* GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp: Added.
(WebKit::nextLogIdentifier):
(WebKit::nullLogger):
(WebKit::RemoteAudioMediaStreamTrackRenderer::RemoteAudioMediaStreamTrackRenderer):
(WebKit::RemoteAudioMediaStreamTrackRenderer::~RemoteAudioMediaStreamTrackRenderer):
(WebKit::RemoteAudioMediaStreamTrackRenderer::storage):
(WebKit::RemoteAudioMediaStreamTrackRenderer::start):
(WebKit::RemoteAudioMediaStreamTrackRenderer::stop):
(WebKit::RemoteAudioMediaStreamTrackRenderer::clear):
(WebKit::RemoteAudioMediaStreamTrackRenderer::setMuted):
(WebKit::RemoteAudioMediaStreamTrackRenderer::setVolume):
(WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesStorageChanged):
(WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesAvailable):
* GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h: Added.
* GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in: Added.
* GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp: Added.
(WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveRendererMessage):
(WebKit::RemoteAudioMediaStreamTrackRendererManager::createRenderer):
(WebKit::RemoteAudioMediaStreamTrackRendererManager::releaseRenderer):
* GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.h: Added.
(WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveMessageFromWebProcess):
* GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in: Added.
* Scripts/webkit/messages.py:
* Sources.txt:
* SourcesCocoa.txt:
* WebKit.xcodeproj/project.pbxproj:
* WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
(WebKit::RemoteMediaPlayerManager::updatePreferences):
* WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp: Added.
(WebKit::AudioMediaStreamTrackRenderer::create):
(WebKit::AudioMediaStreamTrackRenderer::AudioMediaStreamTrackRenderer):
(WebKit::AudioMediaStreamTrackRenderer::~AudioMediaStreamTrackRenderer):
(WebKit::AudioMediaStreamTrackRenderer::start):
(WebKit::AudioMediaStreamTrackRenderer::stop):
(WebKit::AudioMediaStreamTrackRenderer::clear):
(WebKit::AudioMediaStreamTrackRenderer::setMuted):
(WebKit::AudioMediaStreamTrackRenderer::setVolume):
(WebKit::AudioMediaStreamTrackRenderer::pushSamples):
(WebKit::AudioMediaStreamTrackRenderer::storageChanged):
* WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h: Added.
(WebKit::AudioMediaStreamTrackRenderer::identifier const):
* WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererIdentifier.h: Added.

LayoutTests:

* gpu-process/TestExpectations:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@254562 268f45cc-cd09-0410-ab3c-d52691b4dbfc

27 files changed:
LayoutTests/ChangeLog
LayoutTests/gpu-process/TestExpectations
Source/WebCore/ChangeLog
Source/WebCore/platform/mediastream/AudioMediaStreamTrackRenderer.h
Source/WebCore/platform/mediastream/AudioTrackPrivateMediaStream.cpp
Source/WebCore/platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.cpp
Source/WebCore/platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.h
Source/WebKit/ChangeLog
Source/WebKit/DerivedSources-input.xcfilelist
Source/WebKit/DerivedSources-output.xcfilelist
Source/WebKit/DerivedSources.make
Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.h [new file with mode: 0644]
Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in [new file with mode: 0644]
Source/WebKit/Scripts/webkit/messages.py
Source/WebKit/Sources.txt
Source/WebKit/SourcesCocoa.txt
Source/WebKit/WebKit.xcodeproj/project.pbxproj
Source/WebKit/WebProcess/GPU/media/RemoteMediaPlayerManager.cpp
Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h [new file with mode: 0644]
Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererIdentifier.h [new file with mode: 0644]

index 301a8c4..71e6324 100644 (file)
@@ -1,3 +1,12 @@
+2020-01-15  youenn fablet  <youenn@apple.com>
+
+        Add support for MediaStream audio track rendering in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=206175
+
+        Reviewed by Eric Carlson.
+
+        * gpu-process/TestExpectations:
+
 2020-01-15  Oriol Brufau  <obrufau@igalia.com>
 
         [css-grid] Exclude implicit grid tracks from the resolved value
index 7de7780..d8f6bb3 100644 (file)
@@ -229,7 +229,6 @@ accessibility/media-element.html [ Skip ]
 fast/block/float/list-marker-is-float-crash.html [ Skip ]
 fast/css/relative-position-replaced-in-table-display-crash.html [ Skip ]
 fast/mediacapturefromelement [ Skip ]
-fast/mediastream [ Skip ]
 fullscreen/full-screen-iframe-legacy.html [ Skip ]
 fullscreen/video-controls-timeline.html [ Skip ]
 http/tests/appcache/video.html [ Skip ]
@@ -243,7 +242,6 @@ http/tests/media/video-preload.html [ Pass ]
 http/tests/navigation/page-cache-mediakeysession.html [ Skip ]
 http/tests/navigation/page-cache-mediastream.html [ Skip ]
 http/wpt/mediarecorder [ Skip ]
-imported/w3c/web-platform-tests/mediacapture-streams [ Skip ]
 imported/w3c/web-platform-tests/mediacapture-fromelement/idlharness.html [ Skip ]
 imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-stop.html [ Skip ]
 imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-error.html [ Skip ]
@@ -260,6 +258,3 @@ http/tests/webgl [ Skip ]
 js/shared-array-buffer-webgl.html [ Skip ]
 
 webgpu [ Skip ]
-
-webrtc [ Skip ]
-platform/mac/webrtc [ Skip ]
index df1ba91..79ad674 100644 (file)
@@ -1,3 +1,28 @@
+2020-01-15  youenn fablet  <youenn@apple.com>
+
+        Add support for MediaStream audio track rendering in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=206175
+
+        Reviewed by Eric Carlson.
+
+        Simplify model to use start/stop instead of setPaused.
+        Simplify and fix issue in computation of the muted state of the renderer.
+        Covered by existing tests run with GPU process enabled and manual testing
+
+        * platform/mediastream/AudioMediaStreamTrackRenderer.h:
+        * platform/mediastream/AudioTrackPrivateMediaStream.cpp:
+        (WebCore::AudioTrackPrivateMediaStream::playInternal):
+        (WebCore::AudioTrackPrivateMediaStream::pause):
+        (WebCore::AudioTrackPrivateMediaStream::audioSamplesAvailable):
+        (WebCore::AudioTrackPrivateMediaStream::updateRendererMutedState):
+        * platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.cpp:
+        (WebCore::AudioMediaStreamTrackRendererCocoa::start):
+        (WebCore::AudioMediaStreamTrackRendererCocoa::stop):
+        (WebCore::AudioMediaStreamTrackRendererCocoa::clear):
+        (WebCore::AudioMediaStreamTrackRendererCocoa::pushSamples):
+        (WebCore::AudioMediaStreamTrackRendererCocoa::render):
+        * platform/mediastream/mac/AudioMediaStreamTrackRendererCocoa.h:
+
 2020-01-15  Oriol Brufau  <obrufau@igalia.com>
 
         [css-grid] Exclude implicit grid tracks from the resolved value
index 800a930..1462e32 100644 (file)
@@ -44,7 +44,7 @@ public:
     static std::unique_ptr<AudioMediaStreamTrackRenderer> create();
     virtual ~AudioMediaStreamTrackRenderer() = default;
 
-    virtual void setPaused(bool) = 0;
+    virtual void start() = 0;
     virtual void stop() = 0;
     virtual void clear() = 0;
     // May be called on a background thread.
index 97d9e69..e00695c 100644 (file)
@@ -77,7 +77,7 @@ void AudioTrackPrivateMediaStream::playInternal()
     m_isPlaying = true;
     m_autoPlay = false;
 
-    m_renderer->setPaused(false);
+    m_renderer->start();
 }
 
 void AudioTrackPrivateMediaStream::play()
@@ -95,7 +95,7 @@ void AudioTrackPrivateMediaStream::pause()
     m_isPlaying = false;
     m_autoPlay = false;
 
-    m_renderer->setPaused(true);
+    m_renderer->stop();
 }
 
 void AudioTrackPrivateMediaStream::setVolume(float volume)
@@ -139,7 +139,7 @@ void AudioTrackPrivateMediaStream::trackEnabledChanged(MediaStreamTrackPrivate&)
 
 void AudioTrackPrivateMediaStream::updateRendererMutedState()
 {
-    m_renderer->setMuted(m_isPlaying && !streamTrack().muted() && !streamTrack().ended() && streamTrack().enabled());
+    m_renderer->setMuted(streamTrack().muted() || streamTrack().ended() || !streamTrack().enabled());
 }
 
 void AudioTrackPrivateMediaStream::trackEnded(MediaStreamTrackPrivate&)
index e75461f..7cd3165 100644 (file)
@@ -43,17 +43,26 @@ AudioMediaStreamTrackRendererCocoa::AudioMediaStreamTrackRendererCocoa() = defau
 
 AudioMediaStreamTrackRendererCocoa::~AudioMediaStreamTrackRendererCocoa() = default;
 
+void AudioMediaStreamTrackRendererCocoa::start()
+{
+    m_shouldPlay = true;
+
+    if (m_dataSource)
+        m_dataSource->setPaused(false);
+}
+
 void AudioMediaStreamTrackRendererCocoa::stop()
 {
-    if (!m_isAudioUnitStarted)
-        return;
+    m_shouldPlay = false;
 
-    if (m_remoteIOUnit)
-        AudioOutputUnitStop(m_remoteIOUnit);
+    if (m_dataSource)
+        m_dataSource->setPaused(true);
 }
 
 void AudioMediaStreamTrackRendererCocoa::clear()
 {
+    m_shouldPlay = false;
+
     if (m_dataSource)
         m_dataSource->setPaused(true);
 
@@ -66,13 +75,7 @@ void AudioMediaStreamTrackRendererCocoa::clear()
     m_dataSource = nullptr;
     m_inputDescription = nullptr;
     m_outputDescription = nullptr;
-}
-
-void AudioMediaStreamTrackRendererCocoa::setPaused(bool value)
-{
-    m_paused = value;
-    if (m_dataSource)
-        m_dataSource->setPaused(value);
+    m_isAudioUnitStarted = false;
 }
 
 AudioComponentInstance AudioMediaStreamTrackRendererCocoa::createAudioUnit(CAAudioStreamDescription& outputDescription)
@@ -143,6 +146,14 @@ AudioComponentInstance AudioMediaStreamTrackRendererCocoa::createAudioUnit(CAAud
 void AudioMediaStreamTrackRendererCocoa::pushSamples(const MediaTime& sampleTime, const PlatformAudioData& audioData, const AudioStreamDescription& description, size_t sampleCount)
 {
     ASSERT(description.platformDescription().type == PlatformDescription::CAAudioStreamBasicType);
+    if (!m_shouldPlay) {
+        if (m_isAudioUnitStarted) {
+            if (m_remoteIOUnit)
+                AudioOutputUnitStop(m_remoteIOUnit);
+            m_isAudioUnitStarted = false;
+        }
+        return;
+    }
 
     if (!m_inputDescription || *m_inputDescription != description) {
         m_isAudioUnitStarted = false;
@@ -199,7 +210,7 @@ void AudioMediaStreamTrackRendererCocoa::pushSamples(const MediaTime& sampleTime
 
 OSStatus AudioMediaStreamTrackRendererCocoa::render(UInt32 sampleCount, AudioBufferList& ioData, UInt32 /*inBusNumber*/, const AudioTimeStamp& timeStamp, AudioUnitRenderActionFlags& actionFlags)
 {
-    if (isMuted() || m_paused || !m_dataSource) {
+    if (isMuted() || !m_shouldPlay || !m_dataSource) {
         AudioSampleBufferList::zeroABL(ioData, static_cast<size_t>(sampleCount * m_outputDescription->bytesPerFrame()));
         actionFlags = kAudioUnitRenderAction_OutputIsSilence;
         return 0;
index a9e5461..154dfa9 100644 (file)
@@ -48,9 +48,9 @@ public:
 private:
     // AudioMediaStreamTrackRenderer
     void pushSamples(const WTF::MediaTime&, const PlatformAudioData&, const AudioStreamDescription&, size_t) final;
+    void start() final;
     void stop() final;
     void clear() final;
-    void setPaused(bool) final;
 
     static OSStatus inputProc(void*, AudioUnitRenderActionFlags*, const AudioTimeStamp*, UInt32 inBusNumber, UInt32 numberOfFrames, AudioBufferList*);
     OSStatus render(UInt32 sampleCount, AudioBufferList&, UInt32 inBusNumber, const AudioTimeStamp&, AudioUnitRenderActionFlags&);
@@ -65,7 +65,7 @@ private:
     // Cross thread members
     RefPtr<AudioSampleDataSource> m_dataSource;
     bool m_isAudioUnitStarted { false };
-    bool m_paused { false };
+    bool m_shouldPlay { false };
 };
 
 }
index 9d40320..419a1d8 100644 (file)
@@ -1,3 +1,67 @@
+2020-01-15  youenn fablet  <youenn@apple.com>
+
+        Add support for MediaStream audio track rendering in GPUProcess
+        https://bugs.webkit.org/show_bug.cgi?id=206175
+
+        Reviewed by Eric Carlson.
+
+        Implement an AudioMediaStreamTrackRenderer at WebKit level by creating a remote renderer in GPUProcess and sending IPC to pass
+        audio data as well as orders (start/stop/setMuted).
+
+        Implement the remote renderer using WebCore audio track renderer.
+
+        Enable WebKit remote renderer F GPU process for media is enabled.
+
+        * DerivedSources-input.xcfilelist:
+        * DerivedSources-output.xcfilelist:
+        * DerivedSources.make:
+        * GPUProcess/GPUConnectionToWebProcess.cpp:
+        (WebKit::GPUConnectionToWebProcess::audioTrackRendererManager):
+        (WebKit::GPUConnectionToWebProcess::didReceiveMessage):
+        * GPUProcess/GPUConnectionToWebProcess.h:
+        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp: Added.
+        (WebKit::nextLogIdentifier):
+        (WebKit::nullLogger):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::RemoteAudioMediaStreamTrackRenderer):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::~RemoteAudioMediaStreamTrackRenderer):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::storage):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::start):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::stop):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::clear):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::setMuted):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::setVolume):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesStorageChanged):
+        (WebKit::RemoteAudioMediaStreamTrackRenderer::audioSamplesAvailable):
+        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h: Added.
+        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in: Added.
+        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp: Added.
+        (WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveRendererMessage):
+        (WebKit::RemoteAudioMediaStreamTrackRendererManager::createRenderer):
+        (WebKit::RemoteAudioMediaStreamTrackRendererManager::releaseRenderer):
+        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.h: Added.
+        (WebKit::RemoteAudioMediaStreamTrackRendererManager::didReceiveMessageFromWebProcess):
+        * GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in: Added.
+        * Scripts/webkit/messages.py:
+        * Sources.txt:
+        * SourcesCocoa.txt:
+        * WebKit.xcodeproj/project.pbxproj:
+        * WebProcess/GPU/media/RemoteMediaPlayerManager.cpp:
+        (WebKit::RemoteMediaPlayerManager::updatePreferences):
+        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp: Added.
+        (WebKit::AudioMediaStreamTrackRenderer::create):
+        (WebKit::AudioMediaStreamTrackRenderer::AudioMediaStreamTrackRenderer):
+        (WebKit::AudioMediaStreamTrackRenderer::~AudioMediaStreamTrackRenderer):
+        (WebKit::AudioMediaStreamTrackRenderer::start):
+        (WebKit::AudioMediaStreamTrackRenderer::stop):
+        (WebKit::AudioMediaStreamTrackRenderer::clear):
+        (WebKit::AudioMediaStreamTrackRenderer::setMuted):
+        (WebKit::AudioMediaStreamTrackRenderer::setVolume):
+        (WebKit::AudioMediaStreamTrackRenderer::pushSamples):
+        (WebKit::AudioMediaStreamTrackRenderer::storageChanged):
+        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h: Added.
+        (WebKit::AudioMediaStreamTrackRenderer::identifier const):
+        * WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererIdentifier.h: Added.
+
 2020-01-14  Commit Queue  <commit-queue@webkit.org>
 
         Unreviewed, rolling out r254548.
index 4c31a88..1a24bbf 100644 (file)
@@ -20,6 +20,8 @@ $(PROJECT_DIR)/GPUProcess/media/RemoteMediaPlayerManagerProxy.messages.in
 $(PROJECT_DIR)/GPUProcess/media/RemoteMediaPlayerProxy.messages.in
 $(PROJECT_DIR)/GPUProcess/media/RemoteMediaResourceManager.messages.in
 $(PROJECT_DIR)/GPUProcess/webrtc/LibWebRTCCodecsProxy.messages.in
+$(PROJECT_DIR)/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in
+$(PROJECT_DIR)/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in
 $(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
 $(PROJECT_DIR)/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in
 $(PROJECT_DIR)/NetworkProcess/Cookies/WebCookieManager.messages.in
index fabdf98..1dc5465 100644 (file)
@@ -120,6 +120,12 @@ $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/PluginProcessProxyMessagesReplies.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/PluginProxyMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/PluginProxyMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/PluginProxyMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererManagerMessageReceiver.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererManagerMessages.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererManagerMessagesReplies.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererMessageReceiver.cpp
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererMessages.h
+$(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteAudioMediaStreamTrackRendererMessagesReplies.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteLayerTreeDrawingAreaProxyMessageReceiver.cpp
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteLayerTreeDrawingAreaProxyMessages.h
 $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2/RemoteLayerTreeDrawingAreaProxyMessagesReplies.h
index 3d4c19b..a19d079 100644 (file)
@@ -140,6 +140,8 @@ MESSAGE_RECEIVERS = \
     PluginProcessConnectionManager \
     PluginProcessProxy \
     PluginProxy \
+    RemoteAudioMediaStreamTrackRendererManager \
+    RemoteAudioMediaStreamTrackRenderer \
     RemoteLayerTreeDrawingAreaProxy \
     RemoteMediaPlayerManager \
     RemoteMediaPlayerManagerProxy \
index 7f73195..46b1349 100644 (file)
@@ -36,6 +36,9 @@
 #include "LibWebRTCCodecsProxy.h"
 #include "LibWebRTCCodecsProxyMessages.h"
 #include "Logging.h"
+#include "RemoteAudioMediaStreamTrackRendererManager.h"
+#include "RemoteAudioMediaStreamTrackRendererManagerMessages.h"
+#include "RemoteAudioMediaStreamTrackRendererMessages.h"
 #include "RemoteLayerTreeDrawingAreaProxyMessages.h"
 #include "RemoteMediaPlayerManagerProxy.h"
 #include "RemoteMediaPlayerManagerProxyMessages.h"
@@ -150,6 +153,16 @@ RemoteMediaRecorderManager& GPUConnectionToWebProcess::mediaRecorderManager()
 
     return *m_remoteMediaRecorderManager;
 }
+
+#if ENABLE(VIDEO_TRACK)
+RemoteAudioMediaStreamTrackRendererManager& GPUConnectionToWebProcess::audioTrackRendererManager()
+{
+    if (!m_audioTrackRendererManager)
+        m_audioTrackRendererManager = makeUnique<RemoteAudioMediaStreamTrackRendererManager>();
+
+    return *m_audioTrackRendererManager;
+}
+#endif
 #endif
 
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
@@ -187,6 +200,16 @@ void GPUConnectionToWebProcess::didReceiveMessage(IPC::Connection& connection, I
         mediaRecorderManager().didReceiveRemoteMediaRecorderMessage(connection, decoder);
         return;
     }
+#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK)
+    if (decoder.messageReceiverName() == Messages::RemoteAudioMediaStreamTrackRendererManager::messageReceiverName()) {
+        audioTrackRendererManager().didReceiveMessageFromWebProcess(connection, decoder);
+        return;
+    }
+    if (decoder.messageReceiverName() == Messages::RemoteAudioMediaStreamTrackRenderer::messageReceiverName()) {
+        audioTrackRendererManager().didReceiveRendererMessage(connection, decoder);
+        return;
+    }
+#endif
 #endif
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
     if (decoder.messageReceiverName() == Messages::LibWebRTCCodecsProxy::messageReceiverName()) {
index f485964..cc2110b 100644 (file)
@@ -40,6 +40,7 @@ namespace WebKit {
 
 class GPUProcess;
 class LibWebRTCCodecsProxy;
+class RemoteAudioMediaStreamTrackRendererManager;
 class RemoteMediaPlayerManagerProxy;
 class RemoteMediaRecorderManager;
 class RemoteMediaResourceManager;
@@ -73,6 +74,9 @@ private:
 #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
     UserMediaCaptureManagerProxy& userMediaCaptureManagerProxy();
     RemoteMediaRecorderManager& mediaRecorderManager();
+#if ENABLE(VIDEO_TRACK)
+    RemoteAudioMediaStreamTrackRendererManager& audioTrackRendererManager();
+#endif
 #endif
 
     // IPC::Connection::Client
@@ -92,6 +96,9 @@ private:
 #if PLATFORM(COCOA) && ENABLE(MEDIA_STREAM)
     std::unique_ptr<UserMediaCaptureManagerProxy> m_userMediaCaptureManagerProxy;
     std::unique_ptr<RemoteMediaRecorderManager> m_remoteMediaRecorderManager;
+#if ENABLE(VIDEO_TRACK)
+    std::unique_ptr<RemoteAudioMediaStreamTrackRendererManager> m_audioTrackRendererManager;
+#endif
 #endif
 #if PLATFORM(COCOA) && USE(LIBWEBRTC)
     std::unique_ptr<LibWebRTCCodecsProxy> m_libWebRTCCodecsProxy;
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp
new file mode 100644 (file)
index 0000000..ed1fac4
--- /dev/null
@@ -0,0 +1,142 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "RemoteAudioMediaStreamTrackRenderer.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "SharedRingBufferStorage.h"
+#include <WebCore/AudioMediaStreamTrackRenderer.h>
+#include <WebCore/CARingBuffer.h>
+#include <WebCore/WebAudioBufferList.h>
+
+using namespace WebCore;
+
+namespace WebKit {
+
+#if !RELEASE_LOG_DISABLED
+static const void* nextLogIdentifier()
+{
+    static uint64_t logIdentifier = cryptographicallyRandomNumber();
+    return reinterpret_cast<const void*>(++logIdentifier);
+}
+
+static RefPtr<Logger>& nullLogger()
+{
+    static NeverDestroyed<RefPtr<Logger>> logger;
+    return logger;
+}
+#endif
+
+RemoteAudioMediaStreamTrackRenderer::RemoteAudioMediaStreamTrackRenderer()
+    : m_renderer(WebCore::AudioMediaStreamTrackRenderer::create())
+    , m_ringBuffer(makeUnique<CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(nullptr)))
+{
+    ASSERT(m_renderer);
+
+#if !RELEASE_LOG_DISABLED
+    if (!nullLogger().get()) {
+        nullLogger() = Logger::create(this);
+        nullLogger()->setEnabled(this, false);
+    }
+    m_renderer->setLogger(*nullLogger(), nextLogIdentifier());
+#endif
+}
+
+RemoteAudioMediaStreamTrackRenderer::~RemoteAudioMediaStreamTrackRenderer()
+{
+    m_renderer->clear();
+}
+
+SharedRingBufferStorage& RemoteAudioMediaStreamTrackRenderer::storage()
+{
+    return static_cast<SharedRingBufferStorage&>(m_ringBuffer->storage());
+}
+
+void RemoteAudioMediaStreamTrackRenderer::start()
+{
+    m_renderer->start();
+}
+
+void RemoteAudioMediaStreamTrackRenderer::stop()
+{
+    m_renderer->stop();
+}
+
+void RemoteAudioMediaStreamTrackRenderer::clear()
+{
+    m_renderer->clear();
+}
+
+void RemoteAudioMediaStreamTrackRenderer::setMuted(bool value)
+{
+    m_renderer->setMuted(value);
+}
+
+void RemoteAudioMediaStreamTrackRenderer::setVolume(float value)
+{
+    m_renderer->setVolume(value);
+}
+
+void RemoteAudioMediaStreamTrackRenderer::audioSamplesStorageChanged(const SharedMemory::Handle& handle, const WebCore::CAAudioStreamDescription& description, uint64_t numberOfFrames)
+{
+    ASSERT(m_ringBuffer);
+    if (!m_ringBuffer)
+        return;
+
+    m_description = description;
+
+    if (handle.isNull()) {
+        m_ringBuffer->deallocate();
+        storage().setReadOnly(false);
+        storage().setStorage(nullptr);
+        return;
+    }
+
+    auto memory = SharedMemory::map(handle, SharedMemory::Protection::ReadOnly);
+    storage().setStorage(WTFMove(memory));
+    storage().setReadOnly(true);
+
+    m_ringBuffer->allocate(description, numberOfFrames);
+}
+
+void RemoteAudioMediaStreamTrackRenderer::audioSamplesAvailable(MediaTime time, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame)
+{
+    ASSERT(m_ringBuffer);
+    if (!m_ringBuffer)
+        return;
+
+    m_ringBuffer->setCurrentFrameBounds(startFrame, endFrame);
+
+    WebAudioBufferList audioData(m_description, numberOfFrames);
+    m_ringBuffer->fetch(audioData.list(), numberOfFrames, time.timeValue());
+
+    m_renderer->pushSamples(time, audioData, m_description, numberOfFrames);
+}
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h b/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h
new file mode 100644 (file)
index 0000000..319cb4a
--- /dev/null
@@ -0,0 +1,73 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "MessageReceiver.h"
+#include "SharedMemory.h"
+#include <WebCore/CAAudioStreamDescription.h>
+#include <wtf/MediaTime.h>
+
+namespace WebCore {
+class AudioMediaStreamTrackRenderer;
+class CARingBuffer;
+}
+
+namespace WebKit {
+
+class GPUConnectionToWebProcess;
+class SharedRingBufferStorage;
+
+class RemoteAudioMediaStreamTrackRenderer final : private IPC::MessageReceiver {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    RemoteAudioMediaStreamTrackRenderer();
+    ~RemoteAudioMediaStreamTrackRenderer();
+
+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
+
+private:
+    // IPC::MessageReceiver
+    void start();
+    void stop();
+    void clear();
+    void setMuted(bool);
+    void setVolume(float);
+    void audioSamplesStorageChanged(const SharedMemory::Handle&, const WebCore::CAAudioStreamDescription&, uint64_t numberOfFrames);
+    void audioSamplesAvailable(MediaTime, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame);
+
+    SharedRingBufferStorage& storage();
+
+    std::unique_ptr<WebCore::AudioMediaStreamTrackRenderer> m_renderer;
+
+    WebCore::CAAudioStreamDescription m_description;
+    std::unique_ptr<WebCore::CARingBuffer> m_ringBuffer;
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in b/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in
new file mode 100644 (file)
index 0000000..9bfecc2
--- /dev/null
@@ -0,0 +1,38 @@
+# Copyright (C) 2019 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+messages -> RemoteAudioMediaStreamTrackRenderer NotRefCounted {
+    Start()
+    Stop()
+    Clear()
+
+    SetMuted(bool value)
+    SetVolume(float value)
+
+    AudioSamplesStorageChanged(WebKit::SharedMemory::Handle storageHandle, WebCore::CAAudioStreamDescription description, uint64_t numberOfFrames)
+    AudioSamplesAvailable(MediaTime time, uint64_t numberOfFrames, uint64_t startFrame, uint64_t endFrame)
+}
+
+#endif
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp b/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp
new file mode 100644 (file)
index 0000000..46833c8
--- /dev/null
@@ -0,0 +1,61 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "RemoteAudioMediaStreamTrackRendererManager.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "Decoder.h"
+#include "RemoteAudioMediaStreamTrackRenderer.h"
+
+namespace WebKit {
+
+RemoteAudioMediaStreamTrackRendererManager::RemoteAudioMediaStreamTrackRendererManager() = default;
+
+RemoteAudioMediaStreamTrackRendererManager::~RemoteAudioMediaStreamTrackRendererManager() = default;
+
+void RemoteAudioMediaStreamTrackRendererManager::didReceiveRendererMessage(IPC::Connection& connection, IPC::Decoder& decoder)
+{
+    if (auto* renderer = m_renderers.get(makeObjectIdentifier<AudioMediaStreamTrackRendererIdentifierType>(decoder.destinationID())))
+        renderer->didReceiveMessage(connection, decoder);
+}
+
+void RemoteAudioMediaStreamTrackRendererManager::createRenderer(AudioMediaStreamTrackRendererIdentifier identifier)
+{
+    ASSERT(!m_renderers.contains(identifier));
+    m_renderers.add(identifier, makeUnique<RemoteAudioMediaStreamTrackRenderer>());
+}
+
+void RemoteAudioMediaStreamTrackRendererManager::releaseRenderer(AudioMediaStreamTrackRendererIdentifier identifier)
+{
+    ASSERT(m_renderers.contains(identifier));
+    m_renderers.remove(identifier);
+}
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.h b/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.h
new file mode 100644 (file)
index 0000000..bede5ae
--- /dev/null
@@ -0,0 +1,64 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "AudioMediaStreamTrackRendererIdentifier.h"
+#include "MessageReceiver.h"
+#include <wtf/Forward.h>
+#include <wtf/HashMap.h>
+
+namespace IPC {
+class Connection;
+class Decoder;
+}
+
+namespace WebKit {
+
+class RemoteAudioMediaStreamTrackRenderer;
+
+class RemoteAudioMediaStreamTrackRendererManager final : private IPC::MessageReceiver {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    RemoteAudioMediaStreamTrackRendererManager();
+    ~RemoteAudioMediaStreamTrackRendererManager();
+
+    void didReceiveRendererMessage(IPC::Connection&, IPC::Decoder&);
+    void didReceiveMessageFromWebProcess(IPC::Connection& connection, IPC::Decoder& decoder) { didReceiveMessage(connection, decoder); }
+
+private:
+    // IPC::MessageReceiver
+    void didReceiveMessage(IPC::Connection&, IPC::Decoder&) final;
+    void createRenderer(AudioMediaStreamTrackRendererIdentifier);
+    void releaseRenderer(AudioMediaStreamTrackRendererIdentifier);
+
+    HashMap<AudioMediaStreamTrackRendererIdentifier, std::unique_ptr<RemoteAudioMediaStreamTrackRenderer>> m_renderers;
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in b/Source/WebKit/GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.messages.in
new file mode 100644 (file)
index 0000000..3a56021
--- /dev/null
@@ -0,0 +1,31 @@
+# Copyright (C) 2019 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+#    notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+#    notice, this list of conditions and the following disclaimer in the
+#    documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+# THE POSSIBILITY OF SUCH DAMAGE.
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+messages -> RemoteAudioMediaStreamTrackRendererManager NotRefCounted {
+    CreateRenderer(WebKit::AudioMediaStreamTrackRendererIdentifier id)
+    ReleaseRenderer(WebKit::AudioMediaStreamTrackRendererIdentifier id)
+}
+
+#endif
index 7b9213d..2564f52 100644 (file)
@@ -218,6 +218,7 @@ def types_that_cannot_be_forward_declared():
         'WebCore::ServiceWorkerRegistrationIdentifier',
         'WebCore::SWServerConnectionIdentifier',
         'WebKit::ActivityStateChangeID',
+        'WebKit::AudioMediaStreamTrackRendererIdentifier',
         'WebKit::LayerHostingContextID',
         'WebKit::MediaPlayerPrivateRemoteIdentifier',
         'WebKit::MediaRecorderIdentifier',
index e6dfc30..472e4e0 100644 (file)
@@ -24,6 +24,8 @@
 GPUProcess/GPUProcess.cpp
 GPUProcess/GPUConnectionToWebProcess.cpp
 GPUProcess/GPUProcessCreationParameters.cpp
+GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp
+GPUProcess/webrtc/RemoteAudioMediaStreamTrackRendererManager.cpp
 GPUProcess/webrtc/RemoteMediaRecorder.cpp
 GPUProcess/webrtc/RemoteMediaRecorderManager.cpp
 GPUProcess/media/RemoteAudioTrackProxy.cpp
@@ -525,6 +527,7 @@ WebProcess/GPU/media/VideoTrackPrivateRemote.cpp
 WebProcess/GPU/webrtc/LibWebRTCCodecs.cpp
 WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
 WebProcess/GPU/webrtc/MediaRecorderProvider.cpp
+WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp
 
 WebProcess/Network/NetworkProcessConnection.cpp
 WebProcess/Network/WebLoaderStrategy.cpp
index 64f85a6..b23832a 100644 (file)
@@ -640,6 +640,8 @@ GPUProcessProxyMessageReceiver.cpp
 GPUProcessMessageReceiver.cpp
 LibWebRTCCodecsProxyMessageReceiver.cpp
 LibWebRTCCodecsMessageReceiver.cpp
+RemoteAudioMediaStreamTrackRendererManagerMessageReceiver.cpp
+RemoteAudioMediaStreamTrackRendererMessageReceiver.cpp
 RemoteMediaRecorderMessageReceiver.cpp
 RemoteMediaRecorderManagerMessageReceiver.cpp
 ServiceWorkerFetchTaskMessageReceiver.cpp
index 2d90eaf..1c70e64 100644 (file)
                41D129D91F3D101400D15E47 /* WebCacheStorageProvider.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = WebCacheStorageProvider.h; sourceTree = "<group>"; };
                41D5C6D8238EB20D00B9B3CB /* ServiceWorkerSoftUpdateLoader.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ServiceWorkerSoftUpdateLoader.h; sourceTree = "<group>"; };
                41D5C6D9238EB20E00B9B3CB /* ServiceWorkerSoftUpdateLoader.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = ServiceWorkerSoftUpdateLoader.cpp; sourceTree = "<group>"; };
+               41D6455223CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.messages.in */ = {isa = PBXFileReference; lastKnownFileType = text; name = RemoteAudioMediaStreamTrackRenderer.messages.in; path = GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.messages.in; sourceTree = SOURCE_ROOT; };
+               41D6455323CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = RemoteAudioMediaStreamTrackRenderer.h; path = GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.h; sourceTree = SOURCE_ROOT; };
+               41D6455423CCB92200486E0E /* RemoteAudioMediaStreamTrackRenderer.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = RemoteAudioMediaStreamTrackRenderer.cpp; path = GPUProcess/webrtc/RemoteAudioMediaStreamTrackRenderer.cpp; sourceTree = SOURCE_ROOT; };
+               41D6455523CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = RemoteAudioMediaStreamTrackRendererManager.messages.in; sourceTree = "<group>"; };
+               41D6455623CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = RemoteAudioMediaStreamTrackRendererManager.cpp; sourceTree = "<group>"; };
+               41D6455723CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RemoteAudioMediaStreamTrackRendererManager.h; sourceTree = "<group>"; };
                41DC45941E3D6E1E00B11F51 /* NetworkRTCProvider.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkRTCProvider.h; sourceTree = "<group>"; };
                41DC45951E3D6E1E00B11F51 /* NetworkRTCProvider.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkRTCProvider.cpp; sourceTree = "<group>"; };
                41DC45981E3D6ED600B11F51 /* NetworkRTCProvider.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = NetworkRTCProvider.messages.in; sourceTree = "<group>"; };
                41FBE822206DA79C000F0741 /* NetworkContentRuleListManager.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = NetworkContentRuleListManager.h; sourceTree = "<group>"; };
                41FBE823206DA79C000F0741 /* NetworkContentRuleListManager.messages.in */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = NetworkContentRuleListManager.messages.in; sourceTree = "<group>"; };
                41FBE824206DA79C000F0741 /* NetworkContentRuleListManager.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = NetworkContentRuleListManager.cpp; sourceTree = "<group>"; };
+               41FCD6AD23CCC26000C62567 /* AudioMediaStreamTrackRenderer.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioMediaStreamTrackRenderer.h; sourceTree = "<group>"; };
+               41FCD6AE23CCC26100C62567 /* AudioMediaStreamTrackRenderer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AudioMediaStreamTrackRenderer.cpp; sourceTree = "<group>"; };
+               41FCD6AF23CCC26100C62567 /* AudioMediaStreamTrackRendererIdentifier.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AudioMediaStreamTrackRendererIdentifier.h; sourceTree = "<group>"; };
                4450AEBF1DC3FAE5009943F2 /* SharedMemoryCocoa.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SharedMemoryCocoa.cpp; sourceTree = "<group>"; };
                4459984122833E6000E61373 /* SyntheticEditingCommandType.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = SyntheticEditingCommandType.h; sourceTree = "<group>"; };
                449D90D821FD63FE00F677C0 /* LocalAuthenticationSoftLink.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = LocalAuthenticationSoftLink.mm; sourceTree = "<group>"; };
                4172198823B6126100AE5686 /* webrtc */ = {
                        isa = PBXGroup;
                        children = (
+                               41FCD6AE23CCC26100C62567 /* AudioMediaStreamTrackRenderer.cpp */,
+                               41FCD6AD23CCC26000C62567 /* AudioMediaStreamTrackRenderer.h */,
+                               41FCD6AF23CCC26100C62567 /* AudioMediaStreamTrackRendererIdentifier.h */,
                                4172198A23B6128200AE5686 /* LibWebRTCCodecs.cpp */,
                                4172198923B6128200AE5686 /* LibWebRTCCodecs.h */,
                                4172198D23B62C7C00AE5686 /* LibWebRTCCodecs.messages.in */,
                                41E0A7C623B6397800561060 /* LibWebRTCCodecsProxy.h */,
                                41E0A7C723B6397900561060 /* LibWebRTCCodecsProxy.messages.in */,
                                41E0A7C823B6397900561060 /* LibWebRTCCodecsProxy.mm */,
+                               41D6455423CCB92200486E0E /* RemoteAudioMediaStreamTrackRenderer.cpp */,
+                               41D6455323CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.h */,
+                               41D6455223CCB92100486E0E /* RemoteAudioMediaStreamTrackRenderer.messages.in */,
+                               41D6455623CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.cpp */,
+                               41D6455723CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.h */,
+                               41D6455523CCB96900486E0E /* RemoteAudioMediaStreamTrackRendererManager.messages.in */,
                                4176E89D23C376D5003E83FE /* RemoteMediaRecorder.cpp */,
                                4176E89823C36677003E83FE /* RemoteMediaRecorder.h */,
                                4176E89C23C37479003E83FE /* RemoteMediaRecorder.messages.in */,
index cadd88a..547d45f 100644 (file)
@@ -28,7 +28,9 @@
 
 #if ENABLE(GPU_PROCESS)
 
+#include "AudioMediaStreamTrackRenderer.h"
 #include "MediaPlayerPrivateRemote.h"
+#include "RemoteAudioMediaStreamTrackRenderer.h"
 #include "RemoteMediaPlayerConfiguration.h"
 #include "RemoteMediaPlayerMIMETypeCache.h"
 #include "RemoteMediaPlayerManagerMessages.h"
@@ -366,6 +368,11 @@ void RemoteMediaPlayerManager::updatePreferences(const Settings& settings)
     };
 
     RemoteMediaPlayerSupport::setRegisterRemotePlayerCallback(settings.useGPUProcessForMedia() ? WTFMove(registerEngine) : RemoteMediaPlayerSupport::RegisterRemotePlayerCallback());
+
+#if PLATFORM(COCOA) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+    if (settings.useGPUProcessForMedia())
+        WebCore::AudioMediaStreamTrackRenderer::setCreator(WebKit::AudioMediaStreamTrackRenderer::create);
+#endif
 }
 
 void RemoteMediaPlayerManager::updateCachedState(MediaPlayerPrivateRemoteIdentifier id, RemoteMediaPlayerState&& state)
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp b/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.cpp
new file mode 100644 (file)
index 0000000..6acd222
--- /dev/null
@@ -0,0 +1,120 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "AudioMediaStreamTrackRenderer.h"
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "GPUProcessConnection.h"
+#include "RemoteAudioMediaStreamTrackRendererManagerMessages.h"
+#include "RemoteAudioMediaStreamTrackRendererMessages.h"
+#include "WebProcess.h"
+#include <WebCore/AudioMediaStreamTrackRenderer.h>
+#include <WebCore/CARingBuffer.h>
+#include <WebCore/WebAudioBufferList.h>
+
+namespace WebKit {
+
+std::unique_ptr<WebCore::AudioMediaStreamTrackRenderer> AudioMediaStreamTrackRenderer::create()
+{
+    return std::unique_ptr<AudioMediaStreamTrackRenderer>(new AudioMediaStreamTrackRenderer { WebProcess::singleton().ensureGPUProcessConnection().connection() });
+}
+
+AudioMediaStreamTrackRenderer::AudioMediaStreamTrackRenderer(Ref<IPC::Connection>&& connection)
+    : m_connection(WTFMove(connection))
+    , m_identifier(AudioMediaStreamTrackRendererIdentifier::generate())
+    , m_ringBuffer(makeUnique<WebCore::CARingBuffer>(makeUniqueRef<SharedRingBufferStorage>(this)))
+{
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRendererManager::CreateRenderer { m_identifier }, 0);
+}
+
+AudioMediaStreamTrackRenderer::~AudioMediaStreamTrackRenderer()
+{
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRendererManager::ReleaseRenderer { m_identifier }, 0);
+}
+
+void AudioMediaStreamTrackRenderer::start()
+{
+    m_isPlaying = true;
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::Start { }, m_identifier);
+}
+
+void AudioMediaStreamTrackRenderer::stop()
+{
+    m_isPlaying = false;
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::Stop { }, m_identifier);
+}
+
+void AudioMediaStreamTrackRenderer::clear()
+{
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::Clear { }, m_identifier);
+}
+
+void AudioMediaStreamTrackRenderer::setMuted(bool value)
+{
+    WebCore::AudioMediaStreamTrackRenderer::setMuted(value);
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::SetMuted { value }, m_identifier);
+}
+
+void AudioMediaStreamTrackRenderer::setVolume(float value)
+{
+    WebCore::AudioMediaStreamTrackRenderer::setVolume(value);
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::SetVolume { value }, m_identifier);
+}
+
+void AudioMediaStreamTrackRenderer::pushSamples(const MediaTime& time, const WebCore::PlatformAudioData& audioData, const WebCore::AudioStreamDescription& description, size_t numberOfFrames)
+{
+    if (!m_isPlaying)
+        return;
+
+    if (m_description != description) {
+        ASSERT(description.platformDescription().type == WebCore::PlatformDescription::CAAudioStreamBasicType);
+        m_description = *WTF::get<const AudioStreamBasicDescription*>(description.platformDescription().description);
+
+        // Allocate a ring buffer large enough to contain 2 seconds of audio.
+        m_numberOfFrames = m_description.sampleRate() * 2;
+        m_ringBuffer->allocate(m_description.streamDescription(), m_numberOfFrames);
+    }
+
+    ASSERT(is<WebCore::WebAudioBufferList>(audioData));
+    m_ringBuffer->store(downcast<WebCore::WebAudioBufferList>(audioData).list(), numberOfFrames, time.timeValue());
+    uint64_t startFrame;
+    uint64_t endFrame;
+    m_ringBuffer->getCurrentFrameBounds(startFrame, endFrame);
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::AudioSamplesAvailable { time, numberOfFrames, startFrame, endFrame }, m_identifier);
+}
+
+void AudioMediaStreamTrackRenderer::storageChanged(SharedMemory* storage)
+{
+    SharedMemory::Handle handle;
+    if (storage)
+        storage->createHandle(handle, SharedMemory::Protection::ReadOnly);
+    m_connection->send(Messages::RemoteAudioMediaStreamTrackRenderer::AudioSamplesStorageChanged { handle, m_description, static_cast<uint64_t>(m_numberOfFrames) }, m_identifier);
+}
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h b/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRenderer.h
new file mode 100644 (file)
index 0000000..177bf7a
--- /dev/null
@@ -0,0 +1,74 @@
+/*
+ * Copyright (C) 2020 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
+
+#include "AudioMediaStreamTrackRendererIdentifier.h"
+#include "SharedRingBufferStorage.h"
+#include <WebCore/AudioMediaStreamTrackRenderer.h>
+#include <wtf/MediaTime.h>
+
+namespace IPC {
+class Connection;
+}
+
+namespace WebKit {
+
+class AudioMediaStreamTrackRenderer final : public WebCore::AudioMediaStreamTrackRenderer, public SharedRingBufferStorage::Client {
+    WTF_MAKE_FAST_ALLOCATED;
+public:
+    static std::unique_ptr<WebCore::AudioMediaStreamTrackRenderer> create();
+    ~AudioMediaStreamTrackRenderer();
+
+    AudioMediaStreamTrackRendererIdentifier identifier() const { return m_identifier; }
+
+private:
+    explicit AudioMediaStreamTrackRenderer(Ref<IPC::Connection>&&);
+
+    // SharedRingBufferStorage::Client
+    void storageChanged(SharedMemory*);
+
+    // WebCore::AudioMediaStreamTrackRenderer
+    void start() final;
+    void stop() final;
+    void clear() final;
+    void setMuted(bool) final;
+    void setVolume(float) final;
+    void pushSamples(const MediaTime&, const WebCore::PlatformAudioData&, const WebCore::AudioStreamDescription&, size_t) final;
+
+    Ref<IPC::Connection> m_connection;
+    AudioMediaStreamTrackRendererIdentifier m_identifier;
+
+    std::unique_ptr<WebCore::CARingBuffer> m_ringBuffer;
+    WebCore::CAAudioStreamDescription m_description { };
+    int64_t m_numberOfFrames { 0 };
+    bool m_isPlaying { false };
+};
+
+}
+
+#endif // PLATFORM(COCOA) && ENABLE(GPU_PROCESS) && ENABLE(VIDEO_TRACK) && ENABLE(MEDIA_STREAM)
diff --git a/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererIdentifier.h b/Source/WebKit/WebProcess/GPU/webrtc/AudioMediaStreamTrackRendererIdentifier.h
new file mode 100644 (file)
index 0000000..612a084
--- /dev/null
@@ -0,0 +1,35 @@
+/*
+ * Copyright (C) 2019 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice , this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#include <wtf/ObjectIdentifier.h>
+
+namespace WebKit {
+
+enum AudioMediaStreamTrackRendererIdentifierType { };
+using AudioMediaStreamTrackRendererIdentifier = ObjectIdentifier<AudioMediaStreamTrackRendererIdentifierType>;
+
+} // namespace WebKit