[MediaStream] Allow ports to optionally do screen capture in the UI process
authoreric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 18 Oct 2018 22:52:15 +0000 (22:52 +0000)
committereric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 18 Oct 2018 22:52:15 +0000 (22:52 +0000)
commiteb5d918868aaa7f6263e8b172f47d7664179048d
treeb958b0489ac845901ac2d864378574ea63f8e578
parent4991dc16eb70a01fa1c0d16db84712c7ac2d3308
[MediaStream] Allow ports to optionally do screen capture in the UI process
https://bugs.webkit.org/show_bug.cgi?id=190728
<rdar://problem/45376824>

Reviewed by Jer Noble and Tim Horton.

Source/WebCore:

No new tests, covered by existing tests.

* Sources.txt: Add RemoteVideoSample.cpp.
* WebCore.xcodeproj/project.pbxproj: Ditto.

* platform/MediaSample.h:
(WebCore::MediaSample::videoPixelFormat const):

* platform/graphics/RemoteVideoSample.cpp: Added.
(WebCore::RemoteVideoSample::~RemoteVideoSample):
(WebCore::RemoteVideoSample::create):
(WebCore::RemoteVideoSample::RemoteVideoSample):
(WebCore::RemoteVideoSample::surface):
* platform/graphics/RemoteVideoSample.h: Added.
(WebCore::RemoteVideoSample::time const):
(WebCore::RemoteVideoSample::videoFormat const):
(WebCore::RemoteVideoSample::size const):
(WebCore::RemoteVideoSample::encode const):
(WebCore::RemoteVideoSample::decode):
* platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h:
* platform/graphics/cv/ImageTransferSessionVT.h:

* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::remoteVideoSampleAvailable): Call observers.
* platform/mediastream/RealtimeMediaSource.h:

* platform/mediastream/RealtimeVideoSource.cpp:
(WebCore::RealtimeVideoSource::dispatchMediaSampleToObservers): Dispatch remote samples without
resizing, resize local samples if necessary.
* platform/mediastream/RealtimeVideoSource.h:

* platform/mediastream/mac/AVVideoCaptureSource.mm:
(WebCore::AVVideoCaptureSource::captureOutputDidOutputSampleBufferFromConnection): Don't resize
samples, it will be done in the base class if necessary.

* platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp:
(WebCore::DisplayCaptureSourceCocoa::emitFrame): Don't resize samples when running in the UI
process, it will be done in the web process.

* platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp: Remove unneeded include.

Source/WebKit:

* Shared/WebProcessCreationParameters.cpp:
(WebKit::WebProcessCreationParameters::encode const): Encode shouldCaptureDisplayInUIProcess.
(WebKit::WebProcessCreationParameters::decode): Decode shouldCaptureDisplayInUIProcess.
* Shared/WebProcessCreationParameters.h:

* UIProcess/API/APIProcessPoolConfiguration.cpp:
(API::ProcessPoolConfiguration::copy): Copy shouldCaptureDisplayInUIProcess.
* UIProcess/API/APIProcessPoolConfiguration.h:

* UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::remoteVideoSampleAvailable):
(WebKit::UserMediaCaptureManagerProxy::createMediaSourceForCaptureDeviceWithConstraints): Remove
RealtimeMediaSource::Type parameter, CaptureDevice has the same information. Deal with display
capture "devices".
* UIProcess/Cocoa/UserMediaCaptureManagerProxy.h:
* UIProcess/Cocoa/UserMediaCaptureManagerProxy.messages.in:

* UIProcess/WebProcessPool.cpp:
(WebKit::WebProcessPool::initializeNewWebProcess): Copy shouldCaptureDisplayInUIProcess.

* WebProcess/cocoa/UserMediaCaptureManager.cpp:
(WebKit::UserMediaCaptureManager::Source::Source): Only allocate a ring buffer for Audio sources.
(WebKit::UserMediaCaptureManager::Source::~Source): Same for deallocate.
(WebKit::UserMediaCaptureManager::Source::storage): m_ringBuffer is a pointer.
(WebKit::UserMediaCaptureManager::Source::setStorage): Ditto.
(WebKit::UserMediaCaptureManager::Source::setRingBufferFrameBounds): Ditto.
(WebKit::UserMediaCaptureManager::Source::audioSamplesAvailable): Ditto.
(WebKit::UserMediaCaptureManager::Source::remoteVideoSampleAvailable): Create a
PixelBuffer-backed media sample and call videoSampleAvailable.
(WebKit::UserMediaCaptureManager::~UserMediaCaptureManager): Clear the audio and display capture
factory overrides.
(WebKit::UserMediaCaptureManager::initialize): Set the audio and display capture factory overrides.
(WebKit::UserMediaCaptureManager::createCaptureSource):
(WebKit::UserMediaCaptureManager::remoteVideoSampleAvailable):
* WebProcess/cocoa/UserMediaCaptureManager.h:
* WebProcess/cocoa/UserMediaCaptureManager.messages.in:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@237272 268f45cc-cd09-0410-ab3c-d52691b4dbfc
28 files changed:
Source/WebCore/ChangeLog
Source/WebCore/Sources.txt
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/platform/MediaSample.h
Source/WebCore/platform/graphics/RemoteVideoSample.cpp [new file with mode: 0644]
Source/WebCore/platform/graphics/RemoteVideoSample.h [new file with mode: 0644]
Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h
Source/WebCore/platform/graphics/cv/ImageTransferSessionVT.h
Source/WebCore/platform/graphics/cv/ImageTransferSessionVT.mm
Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
Source/WebCore/platform/mediastream/RealtimeMediaSource.h
Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp
Source/WebCore/platform/mediastream/RealtimeVideoSource.h
Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm
Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp
Source/WebCore/platform/mediastream/mac/RealtimeOutgoingVideoSourceCocoa.cpp
Source/WebKit/ChangeLog
Source/WebKit/Shared/WebProcessCreationParameters.cpp
Source/WebKit/Shared/WebProcessCreationParameters.h
Source/WebKit/UIProcess/API/APIProcessPoolConfiguration.cpp
Source/WebKit/UIProcess/API/APIProcessPoolConfiguration.h
Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp
Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h
Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.messages.in
Source/WebKit/UIProcess/WebProcessPool.cpp
Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp
Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.h
Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.messages.in