[MediaStream] Add Mock screen capture source
authoreric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Jan 2018 20:12:27 +0000 (20:12 +0000)
committereric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Jan 2018 20:12:27 +0000 (20:12 +0000)
https://bugs.webkit.org/show_bug.cgi?id=181291
<rdar://problem/36298164>

Reviewed by Dean Jackson.

Source/WebCore:

Tests:  http/tests/media/media-stream/get-display-media-prompt.html
        GetDisplayMediaTest.BasicPrompt
        GetDisplayMediaTest.Constraints

* Modules/mediastream/MediaDevices.cpp:
(WebCore::MediaDevices::MediaDevices): Add static_assert to ensure MediaDevices::DisplayCaptureSurfaceType
and RealtimeMediaSourceSettings::DisplaySurfaceType values are equivalent.
(WebCore::MediaDevices::getSupportedConstraints): Remove bogus code.
* Modules/mediastream/MediaDevices.h: Add DisplayCaptureSurfaceType.
* Modules/mediastream/MediaDevices.idl: Ditto.

* Modules/mediastream/MediaStreamTrack.cpp:
(WebCore::MediaStreamTrack::getSettings const): Add a FIXME.
* Modules/mediastream/MediaStreamTrack.h: Add displaySurface and logicalSurface.

* Modules/mediastream/MediaTrackSupportedConstraints.h: Remove displaySurface and logicalSurface.
* Modules/mediastream/MediaTrackSupportedConstraints.idl:

* SourcesCocoa.txt: Add DisplayCaptureManagerCocoa.cpp and DisplayCaptureSourceCocoa.cpp.

* WebCore.xcodeproj/project.pbxproj: Ditto.

* platform/mediastream/CaptureDevice.h:
(WebCore::CaptureDevice::encode const): Add.
(WebCore::CaptureDevice::decode):

* platform/mediastream/RealtimeMediaSourceCenter.cpp:
(WebCore::RealtimeMediaSourceCenter::getMediaStreamDevices): Include display capture "devices".
(WebCore::RealtimeMediaSourceCenter::validateRequestConstraints): Deal with display capture devices.
(WebCore::RealtimeMediaSourceCenter::captureDeviceWithPersistentID): Ditto.
* platform/mediastream/RealtimeMediaSourceCenter.h:

* platform/mediastream/RealtimeMediaSourceSettings.h:
(WebCore::RealtimeMediaSourceSettings::displaySurface const): Return a DisplaySurfaceType.
(WebCore::RealtimeMediaSourceSettings::setDisplaySurface): Take a DisplaySurfaceType.

* platform/mediastream/mac/DisplayCaptureManagerCocoa.cpp:
(WebCore::DisplayCaptureManagerCocoa::singleton):
(WebCore::DisplayCaptureManagerCocoa::~DisplayCaptureManagerCocoa):
(WebCore::DisplayCaptureManagerCocoa::captureDevices):
(WebCore::DisplayCaptureManagerCocoa::screenCaptureDeviceWithPersistentID):
(WebCore::DisplayCaptureManagerCocoa::captureDeviceWithPersistentID):
* platform/mediastream/mac/DisplayCaptureManagerCocoa.h:

* platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp: Added.
(WebCore::DisplayCaptureSourceCocoa::DisplayCaptureSourceCocoa):
(WebCore::DisplayCaptureSourceCocoa::~DisplayCaptureSourceCocoa):
(WebCore::DisplayCaptureSourceCocoa::capabilities const):
(WebCore::DisplayCaptureSourceCocoa::settings const):
(WebCore::DisplayCaptureSourceCocoa::settingsDidChange):
(WebCore::DisplayCaptureSourceCocoa::startProducingData):
(WebCore::DisplayCaptureSourceCocoa::stopProducingData):
(WebCore::DisplayCaptureSourceCocoa::elapsedTime):
(WebCore::DisplayCaptureSourceCocoa::applyFrameRate):
(WebCore::DisplayCaptureSourceCocoa::emitFrame):
* platform/mediastream/mac/DisplayCaptureSourceCocoa.h:

* platform/mediastream/mac/RealtimeMediaSourceCenterMac.cpp:
(WebCore::RealtimeMediaSourceCenterMac::displayCaptureDeviceManager): New.
* platform/mediastream/mac/RealtimeMediaSourceCenterMac.h:

* platform/mock/MockRealtimeMediaSource.cpp:
(WebCore::deviceMap): Add screen capture "devices".
(WebCore::MockRealtimeMediaSource::displayDevices): New.
* platform/mock/MockRealtimeMediaSource.h:

* platform/mock/MockRealtimeMediaSourceCenter.cpp: Clean up includes.
* platform/mock/MockRealtimeMediaSourceCenter.h:

* platform/mock/MockRealtimeVideoSource.cpp:
(WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource): Mock two screen devices.
(WebCore::MockRealtimeVideoSource::updateSettings): Deal with mock screens.
(WebCore::MockRealtimeVideoSource::initializeCapabilities): Ditto.
(WebCore::MockRealtimeVideoSource::initializeSupportedConstraints): Ditto.
(WebCore::MockRealtimeVideoSource::drawText): Ditto.
(WebCore::MockRealtimeVideoSource::generateFrame): Ditto.
* platform/mock/MockRealtimeVideoSource.h:
(WebCore::MockRealtimeVideoSource::mockCamera const):
(WebCore::MockRealtimeVideoSource::mockScreen const):

Source/WebKit:

* Shared/WebCoreArgumentCoders.cpp:
(IPC::ArgumentCoder<MediaConstraints>::decode):
(IPC::ArgumentCoder<CaptureDevice>::encode): Deleted, moved to CaptureDevice.h
(IPC::ArgumentCoder<CaptureDevice>::decode): Ditto.
* Shared/WebCoreArgumentCoders.h:

* UIProcess/API/Cocoa/WKWebViewPrivate.h: Add _WKCaptureDeviceDisplay.
* UIProcess/Cocoa/UIDelegate.mm:
(WebKit::requestUserMediaAuthorizationForDevices): Deal with display capture.
(WebKit::UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest): Ditto.

* UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
(WebKit::UserMediaPermissionRequestManagerProxy::userMediaAccessWasDenied): requiresAudio -> requiresAudioCapture.
(WebKit::UserMediaPermissionRequestManagerProxy::searchForGrantedRequest const): Never reuse
a previously granted display capture request.

* UIProcess/UserMediaPermissionRequestProxy.cpp:
(WebKit::UserMediaPermissionRequestProxy::allow): Search the eligible devices instead of asking
the source center to find devices.
* UIProcess/UserMediaPermissionRequestProxy.h:
(WebKit::UserMediaPermissionRequestProxy::requiresAudioCapture const): Renamed.
(WebKit::UserMediaPermissionRequestProxy::requiresVideoCapture const): Ditto.
(WebKit::UserMediaPermissionRequestProxy::requiresDisplayCapture const): New.
(WebKit::UserMediaPermissionRequestProxy::requiresAudio const): Deleted.
(WebKit::UserMediaPermissionRequestProxy::requiresVideo const): Deleted.

Tools:

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: Add new test.
* TestWebKitAPI/Tests/WebKitCocoa/GetDisplayMedia.mm:
* TestWebKitAPI/Tests/WebKit/getDisplayMedia.html:

LayoutTests:

* http/tests/media/media-stream/get-display-media-prompt-expected.txt: Added.
* http/tests/media/media-stream/get-display-media-prompt.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@226412 268f45cc-cd09-0410-ab3c-d52691b4dbfc

41 files changed:
LayoutTests/ChangeLog
LayoutTests/http/tests/media/media-stream/get-display-media-prompt-expected.txt [new file with mode: 0644]
LayoutTests/http/tests/media/media-stream/get-display-media-prompt.html [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/Modules/mediastream/MediaDevices.cpp
Source/WebCore/Modules/mediastream/MediaDevices.h
Source/WebCore/Modules/mediastream/MediaDevices.idl
Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp
Source/WebCore/Modules/mediastream/MediaStreamTrack.h
Source/WebCore/Modules/mediastream/MediaTrackSupportedConstraints.h
Source/WebCore/Modules/mediastream/MediaTrackSupportedConstraints.idl
Source/WebCore/SourcesCocoa.txt
Source/WebCore/WebCore.xcodeproj/project.pbxproj
Source/WebCore/platform/mediastream/CaptureDevice.h
Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.cpp
Source/WebCore/platform/mediastream/RealtimeMediaSourceCenter.h
Source/WebCore/platform/mediastream/RealtimeMediaSourceSettings.h
Source/WebCore/platform/mediastream/mac/DisplayCaptureManagerCocoa.cpp [new file with mode: 0644]
Source/WebCore/platform/mediastream/mac/DisplayCaptureManagerCocoa.h [new file with mode: 0644]
Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp [new file with mode: 0644]
Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h [new file with mode: 0644]
Source/WebCore/platform/mediastream/mac/RealtimeMediaSourceCenterMac.cpp
Source/WebCore/platform/mediastream/mac/RealtimeMediaSourceCenterMac.h
Source/WebCore/platform/mock/MockRealtimeMediaSource.cpp
Source/WebCore/platform/mock/MockRealtimeMediaSource.h
Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp
Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.h
Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp
Source/WebCore/platform/mock/MockRealtimeVideoSource.h
Source/WebKit/ChangeLog
Source/WebKit/Shared/WebCoreArgumentCoders.cpp
Source/WebKit/Shared/WebCoreArgumentCoders.h
Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h
Source/WebKit/UIProcess/Cocoa/UIDelegate.mm
Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp
Source/WebKit/UIProcess/UserMediaPermissionRequestProxy.cpp
Source/WebKit/UIProcess/UserMediaPermissionRequestProxy.h
Tools/ChangeLog
Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
Tools/TestWebKitAPI/Tests/WebKit/getDisplayMedia.html [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKitCocoa/GetDisplayMedia.mm [new file with mode: 0644]

index f2ede13d7e90eaaaace8cb296a0a377d0164c695..e4f396c88f8531a53ceea6b1a1e99e2f8d827c62 100644 (file)
@@ -1,3 +1,14 @@
+2018-01-04  Eric Carlson  <eric.carlson@apple.com>
+
+        [MediaStream] Add Mock screen capture source
+        https://bugs.webkit.org/show_bug.cgi?id=181291
+        <rdar://problem/36298164>
+
+        Reviewed by Dean Jackson.
+
+        * http/tests/media/media-stream/get-display-media-prompt-expected.txt: Added.
+        * http/tests/media/media-stream/get-display-media-prompt.html: Added.
+
 2018-01-04  John Wilander  <wilander@apple.com>
 
         Storage Access API: Remove JavaScript confirm() prompt from Document::requestStorageAccess()
diff --git a/LayoutTests/http/tests/media/media-stream/get-display-media-prompt-expected.txt b/LayoutTests/http/tests/media/media-stream/get-display-media-prompt-expected.txt
new file mode 100644 (file)
index 0000000..878a4fc
--- /dev/null
@@ -0,0 +1,32 @@
+Test basic getDisplayMedia prompting behavior
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS numberOfTimesGetUserMediaPromptHasBeenCalled() is 0
+
+** Request an audio-only stream, the user should be prompted **
+PASS numberOfTimesGetUserMediaPromptHasBeenCalled() is 1
+PASS stream.getAudioTracks().length is 1
+PASS stream.getVideoTracks().length is 0
+
+** Request an video-only stream, the user should be prompted **
+PASS numberOfTimesGetUserMediaPromptHasBeenCalled() is 2
+PASS stream.getAudioTracks().length is 0
+PASS stream.getVideoTracks().length is 1
+
+** Request a stream with audio and video, the user should be prompted **
+PASS numberOfTimesGetUserMediaPromptHasBeenCalled() is 3
+PASS stream.getAudioTracks().length is 1
+PASS stream.getVideoTracks().length is 1
+
+** Request a stream with invalid constraints, the user should not be prompted **
+PASS numberOfTimesGetUserMediaPromptHasBeenCalled() is 3
+PASS stream is null
+PASS err instanceof Error  is true
+PASS err.name is "InvalidAccessError"
+
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/http/tests/media/media-stream/get-display-media-prompt.html b/LayoutTests/http/tests/media/media-stream/get-display-media-prompt.html
new file mode 100644 (file)
index 0000000..b445821
--- /dev/null
@@ -0,0 +1,80 @@
+<!DOCTYPE html>
+<html>
+    <head>
+        <title>getDisplayMedia prompt</title>
+        <script src="../../../../resources/js-test-pre.js"></script>
+    </head>
+    <body>
+    <p id="description"></p>
+    <div id="console"></div>
+
+<script>
+
+    let stream;
+    let err;
+    
+    function numberOfTimesGetUserMediaPromptHasBeenCalled() {
+        return testRunner.userMediaPermissionRequestCountForOrigin(document.location.href, document.location.href);
+    }
+    
+    async function promptForAudioOnly() {
+        debug("<br>** Request an audio-only stream, the user should be prompted **");
+        stream = await navigator.mediaDevices.getDisplayMedia({ audio: true });
+        shouldBe("numberOfTimesGetUserMediaPromptHasBeenCalled()", "1");
+        shouldBe("stream.getAudioTracks().length", "1");
+        shouldBe("stream.getVideoTracks().length", "0");
+    }
+
+    async function promptForVideoOnly() {
+        debug("<br>** Request an video-only stream, the user should be prompted **");
+        stream = await navigator.mediaDevices.getDisplayMedia({ video: true });
+        shouldBe("numberOfTimesGetUserMediaPromptHasBeenCalled()", "2");
+        shouldBe("stream.getAudioTracks().length", "0");
+        shouldBe("stream.getVideoTracks().length", "1");
+    }
+
+    async function promptForAudioAndVideo() {
+        debug("<br>** Request a stream with audio and video, the user should be prompted **");
+        stream = await navigator.mediaDevices.getDisplayMedia({ video: true, audio: true });
+        shouldBe("numberOfTimesGetUserMediaPromptHasBeenCalled()", "3");
+        shouldBe("stream.getAudioTracks().length", "1");
+        shouldBe("stream.getVideoTracks().length", "1");
+    }
+    
+    async function promptWithMediaTrackConstraints() {
+        debug("<br>** Request a stream with invalid constraints, the user should not be prompted **");
+        stream = null;
+        try {
+            stream = await navigator.mediaDevices.getDisplayMedia({ video: {width: {exact: 640}, height: {exact: 480}} });
+        } catch (e) {
+            err = e;
+        }
+        
+        shouldBe("numberOfTimesGetUserMediaPromptHasBeenCalled()", "3");
+        shouldBeNull("stream");
+        shouldBeTrue("err instanceof Error "); 
+        shouldBeEqualToString("err.name", "InvalidAccessError");
+    }
+
+    (async function() {
+        description('Test basic getDisplayMedia prompting behavior');
+        jsTestIsAsync = true;
+
+        testRunner.resetUserMediaPermissionRequestCountForOrigin(document.location.href, document.location.href);
+        window.internals.settings.setScreenCaptureEnabled(true);
+
+        shouldBe("numberOfTimesGetUserMediaPromptHasBeenCalled()", "0");
+
+        await promptForAudioOnly();
+        await promptForVideoOnly();
+        await promptForAudioAndVideo();
+        await promptWithMediaTrackConstraints();
+
+        debug("");
+        finishJSTest();
+    })()
+
+</script>
+<script src="../../../../resources/js-test-post.js"></script>
+</body>
+</html>
\ No newline at end of file
index e781797b11ccef0e3884c875b08845a13a425138..140103e3bd3cb762fac0a8bf008323d1d184dfa3 100644 (file)
@@ -1,3 +1,91 @@
+2018-01-04  Eric Carlson  <eric.carlson@apple.com>
+
+        [MediaStream] Add Mock screen capture source
+        https://bugs.webkit.org/show_bug.cgi?id=181291
+        <rdar://problem/36298164>
+
+        Reviewed by Dean Jackson.
+
+        Tests:  http/tests/media/media-stream/get-display-media-prompt.html
+                GetDisplayMediaTest.BasicPrompt
+                GetDisplayMediaTest.Constraints
+
+        * Modules/mediastream/MediaDevices.cpp:
+        (WebCore::MediaDevices::MediaDevices): Add static_assert to ensure MediaDevices::DisplayCaptureSurfaceType
+        and RealtimeMediaSourceSettings::DisplaySurfaceType values are equivalent.
+        (WebCore::MediaDevices::getSupportedConstraints): Remove bogus code.
+        * Modules/mediastream/MediaDevices.h: Add DisplayCaptureSurfaceType.
+        * Modules/mediastream/MediaDevices.idl: Ditto.
+
+        * Modules/mediastream/MediaStreamTrack.cpp:
+        (WebCore::MediaStreamTrack::getSettings const): Add a FIXME.
+        * Modules/mediastream/MediaStreamTrack.h: Add displaySurface and logicalSurface.
+
+        * Modules/mediastream/MediaTrackSupportedConstraints.h: Remove displaySurface and logicalSurface.
+        * Modules/mediastream/MediaTrackSupportedConstraints.idl:
+
+        * SourcesCocoa.txt: Add DisplayCaptureManagerCocoa.cpp and DisplayCaptureSourceCocoa.cpp.
+
+        * WebCore.xcodeproj/project.pbxproj: Ditto.
+
+        * platform/mediastream/CaptureDevice.h:
+        (WebCore::CaptureDevice::encode const): Add.
+        (WebCore::CaptureDevice::decode):
+
+        * platform/mediastream/RealtimeMediaSourceCenter.cpp:
+        (WebCore::RealtimeMediaSourceCenter::getMediaStreamDevices): Include display capture "devices".
+        (WebCore::RealtimeMediaSourceCenter::validateRequestConstraints): Deal with display capture devices.
+        (WebCore::RealtimeMediaSourceCenter::captureDeviceWithPersistentID): Ditto.
+        * platform/mediastream/RealtimeMediaSourceCenter.h:
+
+        * platform/mediastream/RealtimeMediaSourceSettings.h:
+        (WebCore::RealtimeMediaSourceSettings::displaySurface const): Return a DisplaySurfaceType.
+        (WebCore::RealtimeMediaSourceSettings::setDisplaySurface): Take a DisplaySurfaceType.
+
+        * platform/mediastream/mac/DisplayCaptureManagerCocoa.cpp:
+        (WebCore::DisplayCaptureManagerCocoa::singleton):
+        (WebCore::DisplayCaptureManagerCocoa::~DisplayCaptureManagerCocoa):
+        (WebCore::DisplayCaptureManagerCocoa::captureDevices):
+        (WebCore::DisplayCaptureManagerCocoa::screenCaptureDeviceWithPersistentID):
+        (WebCore::DisplayCaptureManagerCocoa::captureDeviceWithPersistentID):
+        * platform/mediastream/mac/DisplayCaptureManagerCocoa.h:
+
+        * platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp: Added.
+        (WebCore::DisplayCaptureSourceCocoa::DisplayCaptureSourceCocoa):
+        (WebCore::DisplayCaptureSourceCocoa::~DisplayCaptureSourceCocoa):
+        (WebCore::DisplayCaptureSourceCocoa::capabilities const):
+        (WebCore::DisplayCaptureSourceCocoa::settings const):
+        (WebCore::DisplayCaptureSourceCocoa::settingsDidChange):
+        (WebCore::DisplayCaptureSourceCocoa::startProducingData):
+        (WebCore::DisplayCaptureSourceCocoa::stopProducingData):
+        (WebCore::DisplayCaptureSourceCocoa::elapsedTime):
+        (WebCore::DisplayCaptureSourceCocoa::applyFrameRate):
+        (WebCore::DisplayCaptureSourceCocoa::emitFrame):
+        * platform/mediastream/mac/DisplayCaptureSourceCocoa.h:
+
+        * platform/mediastream/mac/RealtimeMediaSourceCenterMac.cpp:
+        (WebCore::RealtimeMediaSourceCenterMac::displayCaptureDeviceManager): New.
+        * platform/mediastream/mac/RealtimeMediaSourceCenterMac.h:
+
+        * platform/mock/MockRealtimeMediaSource.cpp:
+        (WebCore::deviceMap): Add screen capture "devices".
+        (WebCore::MockRealtimeMediaSource::displayDevices): New.
+        * platform/mock/MockRealtimeMediaSource.h:
+
+        * platform/mock/MockRealtimeMediaSourceCenter.cpp: Clean up includes.
+        * platform/mock/MockRealtimeMediaSourceCenter.h:
+
+        * platform/mock/MockRealtimeVideoSource.cpp:
+        (WebCore::MockRealtimeVideoSource::MockRealtimeVideoSource): Mock two screen devices.
+        (WebCore::MockRealtimeVideoSource::updateSettings): Deal with mock screens.
+        (WebCore::MockRealtimeVideoSource::initializeCapabilities): Ditto.
+        (WebCore::MockRealtimeVideoSource::initializeSupportedConstraints): Ditto.
+        (WebCore::MockRealtimeVideoSource::drawText): Ditto.
+        (WebCore::MockRealtimeVideoSource::generateFrame): Ditto.
+        * platform/mock/MockRealtimeVideoSource.h:
+        (WebCore::MockRealtimeVideoSource::mockCamera const):
+        (WebCore::MockRealtimeVideoSource::mockScreen const):
+
 2018-01-04  Youenn Fablet  <youenn@apple.com>
 
         FetchResponse should set its internal response text encoding name
index d19bc77f5b2d97f7047e557444d07999a0fde8ca..0d82fc354e8a115fa8c273353e7e7d41bc4d3684 100644 (file)
@@ -39,6 +39,7 @@
 #include "EventNames.h"
 #include "MediaDevicesRequest.h"
 #include "MediaTrackSupportedConstraints.h"
+#include "RealtimeMediaSourceSettings.h"
 #include "RuntimeEnabledFeatures.h"
 #include "UserMediaRequest.h"
 #include <wtf/RandomNumber.h>
@@ -59,6 +60,11 @@ inline MediaDevices::MediaDevices(Document& document)
         if (!m_scheduledEventTimer.isActive())
             m_scheduledEventTimer.startOneShot(Seconds(randomNumber() / 2));
     });
+
+    static_assert(static_cast<size_t>(MediaDevices::DisplayCaptureSurfaceType::Monitor) == static_cast<size_t>(RealtimeMediaSourceSettings::DisplaySurfaceType::Monitor), "MediaDevices::DisplayCaptureSurfaceType::Monitor is not equal to RealtimeMediaSourceSettings::DisplaySurfaceType::Monitor as expected");
+    static_assert(static_cast<size_t>(MediaDevices::DisplayCaptureSurfaceType::Window) == static_cast<size_t>(RealtimeMediaSourceSettings::DisplaySurfaceType::Window), "MediaDevices::DisplayCaptureSurfaceType::Window is not RealtimeMediaSourceSettings::DisplaySurfaceType::Window as expected");
+    static_assert(static_cast<size_t>(MediaDevices::DisplayCaptureSurfaceType::Application) == static_cast<size_t>(RealtimeMediaSourceSettings::DisplaySurfaceType::Application), "MediaDevices::DisplayCaptureSurfaceType::Application is not RealtimeMediaSourceSettings::DisplaySurfaceType::Application as expected");
+    static_assert(static_cast<size_t>(MediaDevices::DisplayCaptureSurfaceType::Browser) == static_cast<size_t>(RealtimeMediaSourceSettings::DisplaySurfaceType::Browser), "MediaDevices::DisplayCaptureSurfaceType::Browser is not RealtimeMediaSourceSettings::DisplaySurfaceType::Browser as expected");
 }
 
 MediaDevices::~MediaDevices()
@@ -145,10 +151,6 @@ MediaTrackSupportedConstraints MediaDevices::getSupportedConstraints()
     result.echoCancellation = supported.supportsEchoCancellation();
     result.deviceId = supported.supportsDeviceId();
     result.groupId = supported.supportsGroupId();
-    if (RuntimeEnabledFeatures::sharedFeatures().screenCaptureEnabled()) {
-        result.deviceId = supported.supportsDeviceId();
-        result.groupId = supported.supportsGroupId();
-    }
 
     return result;
 }
index 3d1396422d668e75ef649fb5015805e6e9f0c707..04cc6059ba36d6fbbdca1ff7c86791b3154bba30 100644 (file)
@@ -60,6 +60,13 @@ public:
     using Promise = DOMPromiseDeferred<IDLInterface<MediaStream>>;
     using EnumerateDevicesPromise = DOMPromiseDeferred<IDLSequence<IDLInterface<MediaDeviceInfo>>>;
 
+    enum class DisplayCaptureSurfaceType {
+        Monitor,
+        Window,
+        Application,
+        Browser,
+    };
+
     struct StreamConstraints {
         Variant<bool, MediaTrackConstraints> video;
         Variant<bool, MediaTrackConstraints> audio;
index 6e476bc90b9d71db8cb1589b2b84a0d9239f947d..31a04de1d4847e5e782c4c4393071c13309c2cd4 100644 (file)
     (boolean or MediaTrackConstraints) video = false;
     (boolean or MediaTrackConstraints) audio = false;
 };
+
+[
+    Conditional=MEDIA_STREAM,
+    EnabledAtRuntime=ScreenCapture
+] enum DisplayCaptureSurfaceType {
+    "monitor",
+    "window",
+    "application",
+    "browser"
+};
index 03215855ea910faa9e278b07d8d27816c26fb38f..86769425f75ed5d51a6de721ab75d28ee5618512 100644 (file)
@@ -170,6 +170,9 @@ MediaStreamTrack::TrackSettings MediaStreamTrack::getSettings(Document& document
         result.deviceId = RealtimeMediaSourceCenter::singleton().hashStringWithSalt(settings.deviceId(), document.deviceIDHashSalt());
     if (settings.supportsGroupId())
         result.groupId = RealtimeMediaSourceCenter::singleton().hashStringWithSalt(settings.groupId(), document.deviceIDHashSalt());
+
+    // FIXME: shouldn't this include displaySurface and logicalSurface?
+
     return result;
 }
 
index ee44162c090bb59faaea70e0122670f4f5d42d2c..73ea57c79e48f067357f6a17a7ea1a0909d50dbd 100644 (file)
@@ -95,6 +95,8 @@ public:
         std::optional<int> sampleRate;
         std::optional<int> sampleSize;
         std::optional<bool> echoCancellation;
+        std::optional<bool> displaySurface;
+        String logicalSurface;
         String deviceId;
         String groupId;
     };
index dfc075f063035ad20fbf2661ff908564d7d5f654..f8626b899568f87642cb079d78d8e1441cc83f25 100644 (file)
@@ -46,8 +46,6 @@ struct MediaTrackSupportedConstraints {
     bool echoCancellation;
     bool deviceId;
     bool groupId;
-    bool displaySurface;
-    bool logicalSurface;
 };
 
 } // namespace WebCore
index e9cb1be0716494db8fcae4bd209dcb13931e76af..5853728b547052dfc0959c6de8c353e99bf72421 100644 (file)
@@ -45,6 +45,4 @@
     // FIXME 169871: add channelCount
     boolean deviceId = true;
     boolean groupId = true;
-    boolean displaySurface = true;
-    boolean logicalSurface = true;
 };
index d28deba06b24411f1d9242a17545efa467ac6694..ec11692847889ac4fa19edfffad39c715b23ea82 100644 (file)
@@ -354,6 +354,8 @@ rendering/RenderThemeCocoa.mm
 rendering/TextAutoSizing.cpp
 
 platform/mediastream/mac/CoreAudioCaptureSource.cpp
+platform/mediastream/mac/DisplayCaptureManagerCocoa.cpp
+platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp
 platform/mediastream/mac/RealtimeIncomingAudioSourceCocoa.cpp
 platform/mediastream/mac/RealtimeIncomingVideoSourceCocoa.cpp
 platform/mediastream/mac/RealtimeMediaSourceCenterMac.cpp
index 40984af7246a1cc539ab2fa07cd27629072e4408..de190a12f1e0309ec25bbaeba356120c9a3f7b7a 100644 (file)
                079F5E4B0F3BEBEA005E0782 /* MediaPlayerPrivate.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaPlayerPrivate.h; sourceTree = "<group>"; };
                07A6D1E91491137700051D0C /* MediaFragmentURIParser.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MediaFragmentURIParser.cpp; sourceTree = "<group>"; };
                07A6D1EA1491137700051D0C /* MediaFragmentURIParser.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MediaFragmentURIParser.h; sourceTree = "<group>"; };
+               07A6D8471FEB700B006441DE /* DisplayCaptureSourceCocoa.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = DisplayCaptureSourceCocoa.cpp; sourceTree = "<group>"; };
+               07A6D8481FEB700B006441DE /* DisplayCaptureManagerCocoa.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = DisplayCaptureManagerCocoa.cpp; sourceTree = "<group>"; };
+               07A6D8491FEB700C006441DE /* DisplayCaptureManagerCocoa.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DisplayCaptureManagerCocoa.h; sourceTree = "<group>"; };
+               07A6D84A1FEB700D006441DE /* DisplayCaptureSourceCocoa.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DisplayCaptureSourceCocoa.h; sourceTree = "<group>"; };
                07AA6B69166D019500D45671 /* InbandTextTrackPrivateAVFObjC.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = InbandTextTrackPrivateAVFObjC.h; sourceTree = "<group>"; };
                07AA6B6A166D019500D45671 /* InbandTextTrackPrivateAVFObjC.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = InbandTextTrackPrivateAVFObjC.mm; sourceTree = "<group>"; };
                07AB996518DA3C010018771E /* RTCConfiguration.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = RTCConfiguration.h; sourceTree = "<group>"; };
                                3F8020341E9E381D00DEC61D /* CoreAudioCaptureDeviceManager.h */,
                                3F3BB5821E709EE400C701F2 /* CoreAudioCaptureSource.cpp */,
                                3F3BB5831E709EE400C701F2 /* CoreAudioCaptureSource.h */,
+                               07A6D8481FEB700B006441DE /* DisplayCaptureManagerCocoa.cpp */,
+                               07A6D8491FEB700C006441DE /* DisplayCaptureManagerCocoa.h */,
+                               07A6D8471FEB700B006441DE /* DisplayCaptureSourceCocoa.cpp */,
+                               07A6D84A1FEB700D006441DE /* DisplayCaptureSourceCocoa.h */,
                                0744ECEB1E0C4AE5000D0944 /* MockRealtimeAudioSourceMac.h */,
                                0744ECEC1E0C4AE5000D0944 /* MockRealtimeAudioSourceMac.mm */,
                                07EE76ED1BEA619800F89133 /* MockRealtimeVideoSourceMac.h */,
index dcc7ce05fa40e331ee5798f38237af4de3c224b7..1e5eaed93cfb45cb4e3f76b38eee06353f98bde9 100644 (file)
@@ -25,6 +25,7 @@
 
 #pragma once
 
+#include <wtf/EnumTraits.h>
 #include <wtf/text/WTFString.h>
 
 namespace WebCore {
@@ -56,6 +57,51 @@ public:
 
     explicit operator bool() const { return m_type != DeviceType::Unknown; }
 
+#if ENABLE(MEDIA_STREAM)
+    template<class Encoder>
+    void encode(Encoder& encoder) const
+    {
+        encoder << m_persistentId;
+        encoder << m_label;
+        encoder << m_groupId;
+        encoder << m_enabled;
+        encoder.encodeEnum(m_type);
+    }
+
+    template <class Decoder>
+    static std::optional<CaptureDevice> decode(Decoder& decoder)
+    {
+        std::optional<String> persistentId;
+        decoder >> persistentId;
+        if (!persistentId)
+            return std::nullopt;
+
+        std::optional<String> label;
+        decoder >> label;
+        if (!label)
+            return std::nullopt;
+
+        std::optional<String> groupId;
+        decoder >> groupId;
+        if (!groupId)
+            return std::nullopt;
+
+        std::optional<bool> enabled;
+        decoder >> enabled;
+        if (!enabled)
+            return std::nullopt;
+
+        std::optional<CaptureDevice::DeviceType> type;
+        decoder >> type;
+        if (!type)
+            return std::nullopt;
+
+        std::optional<CaptureDevice> device = {{ WTFMove(*persistentId), WTFMove(*type), WTFMove(*label), WTFMove(*groupId) }};
+        device->setEnabled(*enabled);
+        return device;
+    }
+#endif
+
 private:
     String m_persistentId;
     DeviceType m_type { DeviceType::Unknown };
@@ -66,3 +112,22 @@ private:
 
 } // namespace WebCore
 
+#if ENABLE(MEDIA_STREAM)
+namespace WTF {
+
+template<> struct EnumTraits<WebCore::CaptureDevice::DeviceType> {
+    using values = EnumValues<
+        WebCore::CaptureDevice::DeviceType,
+        WebCore::CaptureDevice::DeviceType::Unknown,
+        WebCore::CaptureDevice::DeviceType::Microphone,
+        WebCore::CaptureDevice::DeviceType::Camera,
+        WebCore::CaptureDevice::DeviceType::Screen,
+        WebCore::CaptureDevice::DeviceType::Application,
+        WebCore::CaptureDevice::DeviceType::Window,
+        WebCore::CaptureDevice::DeviceType::Browser
+    >;
+};
+
+} // namespace WTF
+#endif
+
index 84434457446fb9c0ae61710b72e493e15d25313b..2efb82d1e58c71d9da98ebca402db81d254bef45 100644 (file)
@@ -140,6 +140,10 @@ Vector<CaptureDevice> RealtimeMediaSourceCenter::getMediaStreamDevices()
         if (device.enabled())
             result.append(device);
     }
+    for (auto& device : displayCaptureDeviceManager().captureDevices()) {
+        if (device.enabled())
+            result.append(device);
+    }
 
     return result;
 }
@@ -248,13 +252,15 @@ void RealtimeMediaSourceCenter::validateRequestConstraints(ValidConstraintsHandl
         String invalidConstraint;
         CaptureSourceOrError sourceOrError;
         switch (device.type()) {
-        case CaptureDevice::DeviceType::Camera:
-            if (request.type == MediaStreamRequest::Type::UserMedia && request.videoConstraints.isValid) {
-                auto sourceOrError = videoFactory().createVideoCaptureSource(device, { });
-                if (sourceOrError && sourceOrError.captureSource->supportsConstraints(request.videoConstraints, invalidConstraint))
-                    videoDeviceInfo.append({sourceOrError.captureSource->fitnessScore(), device});
-            }
+        case CaptureDevice::DeviceType::Camera: {
+            if (request.type != MediaStreamRequest::Type::UserMedia || !request.videoConstraints.isValid)
+                continue;
+
+            auto sourceOrError = videoFactory().createVideoCaptureSource(device, { });
+            if (sourceOrError && sourceOrError.captureSource->supportsConstraints(request.videoConstraints, invalidConstraint))
+                videoDeviceInfo.append({sourceOrError.captureSource->fitnessScore(), device});
             break;
+        }
         case CaptureDevice::DeviceType::Microphone:
             if (request.audioConstraints.isValid) {
                 auto sourceOrError = audioFactory().createAudioCaptureSource(device, { });
@@ -265,11 +271,19 @@ void RealtimeMediaSourceCenter::validateRequestConstraints(ValidConstraintsHandl
         case CaptureDevice::DeviceType::Screen:
         case CaptureDevice::DeviceType::Application:
         case CaptureDevice::DeviceType::Window:
-        case CaptureDevice::DeviceType::Browser:
-            ASSERT(request.type == MediaStreamRequest::Type::DisplayMedia);
-            ASSERT(request.videoConstraints.mandatoryConstraints.isEmpty());
+        case CaptureDevice::DeviceType::Browser: {
+            if (request.type != MediaStreamRequest::Type::DisplayMedia)
+                continue;
+            ASSERT(request.audioConstraints.mandatoryConstraints.isEmpty());
             ASSERT(request.videoConstraints.advancedConstraints.isEmpty());
+            if (!request.videoConstraints.isValid || !request.videoConstraints.advancedConstraints.isEmpty() || !request.videoConstraints.mandatoryConstraints.isEmpty())
+                continue;
+
+            auto sourceOrError = videoFactory().createVideoCaptureSource(device, { });
+            if (sourceOrError && sourceOrError.captureSource->supportsConstraints(request.videoConstraints, invalidConstraint))
+                videoDeviceInfo.append({sourceOrError.captureSource->fitnessScore(), device});
             break;
+        }
         case CaptureDevice::DeviceType::Unknown:
             ASSERT_NOT_REACHED();
             break;
@@ -321,6 +335,8 @@ std::optional<CaptureDevice> RealtimeMediaSourceCenter::captureDeviceWithPersist
     case CaptureDevice::DeviceType::Application:
     case CaptureDevice::DeviceType::Window:
     case CaptureDevice::DeviceType::Browser:
+        return displayCaptureDeviceManager().captureDeviceWithPersistentID(type, id);
+        break;
     case CaptureDevice::DeviceType::Unknown:
         ASSERT_NOT_REACHED();
         break;
index 662b9062a66774a3daa993ed8acf49b4e9e0eca0..0c725f9809d6be97393ccb33dff7d81916598f9d 100644 (file)
@@ -79,6 +79,7 @@ public:
 
     virtual CaptureDeviceManager& audioCaptureDeviceManager() = 0;
     virtual CaptureDeviceManager& videoCaptureDeviceManager() = 0;
+    virtual CaptureDeviceManager& displayCaptureDeviceManager() = 0;
 
     String hashStringWithSalt(const String& id, const String& hashSalt);
     WEBCORE_EXPORT CaptureDevice captureDeviceWithUniqueID(const String& id, const String& hashSalt);
index d69862d4bd866d80b3cb7c6fc0675fe034062529..01c1523b820eb9460892130d96fdbbcad653db32 100644 (file)
@@ -89,9 +89,17 @@ public:
     const AtomicString& groupId() const { return m_groupId; }
     void setGroupId(const AtomicString& groupId) { m_groupId = groupId; }
 
+    enum class DisplaySurfaceType {
+        Monitor,
+        Window,
+        Application,
+        Browser,
+        Invalid,
+    };
+
     bool supportsDisplaySurface() const { return m_supportedConstraints.supportsDisplaySurface(); }
-    const AtomicString& displaySurface() const { return m_displaySurface; }
-    void setDisplaySurface(const AtomicString& displaySurface) { m_displaySurface = displaySurface; }
+    DisplaySurfaceType displaySurface() const { return m_displaySurface; }
+    void setDisplaySurface(DisplaySurfaceType displaySurface) { m_displaySurface = displaySurface; }
 
     bool supportsLogicalSurface() const { return m_supportedConstraints.supportsLogicalSurface(); }
     bool logicalSurface() const { return m_logicalSurface; }
@@ -121,7 +129,7 @@ private:
     AtomicString m_groupId;
     AtomicString m_label;
 
-    AtomicString m_displaySurface;
+    DisplaySurfaceType m_displaySurface { DisplaySurfaceType::Invalid };
     bool m_logicalSurface { 0 };
 
     RealtimeMediaSourceSupportedConstraints m_supportedConstraints;
diff --git a/Source/WebCore/platform/mediastream/mac/DisplayCaptureManagerCocoa.cpp b/Source/WebCore/platform/mediastream/mac/DisplayCaptureManagerCocoa.cpp
new file mode 100644 (file)
index 0000000..c820d30
--- /dev/null
@@ -0,0 +1,81 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DisplayCaptureManagerCocoa.h"
+
+#if ENABLE(MEDIA_STREAM)
+
+#include "Logging.h"
+#include <wtf/NeverDestroyed.h>
+
+namespace WebCore {
+
+DisplayCaptureManagerCocoa& DisplayCaptureManagerCocoa::singleton()
+{
+    static NeverDestroyed<DisplayCaptureManagerCocoa> manager;
+    return manager.get();
+}
+
+DisplayCaptureManagerCocoa::~DisplayCaptureManagerCocoa()
+{
+}
+
+const Vector<CaptureDevice>& DisplayCaptureManagerCocoa::captureDevices()
+{
+    return m_displays;
+}
+
+std::optional<CaptureDevice> DisplayCaptureManagerCocoa::screenCaptureDeviceWithPersistentID(const String& deviceID)
+{
+    UNUSED_PARAM(deviceID);
+    return std::nullopt;
+}
+
+std::optional<CaptureDevice> DisplayCaptureManagerCocoa::captureDeviceWithPersistentID(CaptureDevice::DeviceType type, const String& id)
+{
+    switch (type) {
+    case CaptureDevice::DeviceType::Screen:
+        return screenCaptureDeviceWithPersistentID(id);
+        break;
+            
+    case CaptureDevice::DeviceType::Application:
+    case CaptureDevice::DeviceType::Window:
+    case CaptureDevice::DeviceType::Browser:
+        break;
+
+    case CaptureDevice::DeviceType::Camera:
+    case CaptureDevice::DeviceType::Microphone:
+    case CaptureDevice::DeviceType::Unknown:
+        ASSERT_NOT_REACHED();
+        break;
+    }
+
+    return std::nullopt;
+}
+
+} // namespace WebCore
+
+#endif // ENABLE(MEDIA_STREAM)
diff --git a/Source/WebCore/platform/mediastream/mac/DisplayCaptureManagerCocoa.h b/Source/WebCore/platform/mediastream/mac/DisplayCaptureManagerCocoa.h
new file mode 100644 (file)
index 0000000..84962f7
--- /dev/null
@@ -0,0 +1,51 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(MEDIA_STREAM)
+
+#include "CaptureDeviceManager.h"
+
+namespace WebCore {
+
+class DisplayCaptureManagerCocoa final : public CaptureDeviceManager {
+public:
+    static DisplayCaptureManagerCocoa& singleton();
+    DisplayCaptureManagerCocoa() = default;
+
+private:
+    virtual ~DisplayCaptureManagerCocoa();
+
+    const Vector<CaptureDevice>& captureDevices() final;
+    std::optional<CaptureDevice> captureDeviceWithPersistentID(CaptureDevice::DeviceType, const String&) final;
+    std::optional<CaptureDevice> screenCaptureDeviceWithPersistentID(const String&);
+
+    Vector<CaptureDevice> m_displays;
+};
+
+} // namespace WebCore
+
+#endif // ENABLE(MEDIA_STREAM)
diff --git a/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp b/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.cpp
new file mode 100644 (file)
index 0000000..261cba7
--- /dev/null
@@ -0,0 +1,148 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "config.h"
+#include "DisplayCaptureSourceCocoa.h"
+
+#if ENABLE(MEDIA_STREAM)
+
+#include "Logging.h"
+#include "RealtimeMediaSource.h"
+#include "RealtimeMediaSourceCenter.h"
+#include "RealtimeMediaSourceSettings.h"
+#include "Timer.h"
+#include <CoreMedia/CMSync.h>
+#include <mach/mach_time.h>
+#include <pal/avfoundation/MediaTimeAVFoundation.h>
+#include <pal/cf/CoreMediaSoftLink.h>
+#include <pal/spi/cf/CoreAudioSPI.h>
+#include <sys/time.h>
+#include <wtf/MainThread.h>
+#include <wtf/NeverDestroyed.h>
+
+namespace WebCore {
+using namespace PAL;
+
+DisplayCaptureSourceCocoa::DisplayCaptureSourceCocoa(const String& name)
+    : RealtimeMediaSource("", Type::Video, name)
+    , m_timer(RunLoop::current(), this, &DisplayCaptureSourceCocoa::emitFrame)
+{
+}
+
+DisplayCaptureSourceCocoa::~DisplayCaptureSourceCocoa()
+{
+#if PLATFORM(IOS)
+    RealtimeMediaSourceCenter::singleton().videoFactory().unsetActiveSource(*this);
+#endif
+}
+
+const RealtimeMediaSourceCapabilities& DisplayCaptureSourceCocoa::capabilities() const
+{
+    if (!m_capabilities) {
+        RealtimeMediaSourceCapabilities capabilities(settings().supportedConstraints());
+
+        // FIXME: what should these be?
+        capabilities.setWidth(CapabilityValueOrRange(72, 2880));
+        capabilities.setHeight(CapabilityValueOrRange(45, 1800));
+        capabilities.setFrameRate(CapabilityValueOrRange(.01, 60.0));
+
+        m_capabilities = WTFMove(capabilities);
+    }
+    return m_capabilities.value();
+}
+
+const RealtimeMediaSourceSettings& DisplayCaptureSourceCocoa::settings() const
+{
+    if (!m_currentSettings) {
+        RealtimeMediaSourceSettings settings;
+        settings.setFrameRate(frameRate());
+        auto size = this->size();
+        if (size.width() && size.height()) {
+            settings.setWidth(size.width());
+            settings.setHeight(size.height());
+        }
+
+        RealtimeMediaSourceSupportedConstraints supportedConstraints;
+        supportedConstraints.setSupportsFrameRate(true);
+        supportedConstraints.setSupportsWidth(true);
+        supportedConstraints.setSupportsHeight(true);
+        supportedConstraints.setSupportsAspectRatio(true);
+        settings.setSupportedConstraints(supportedConstraints);
+
+        m_currentSettings = WTFMove(settings);
+    }
+    return m_currentSettings.value();
+}
+
+void DisplayCaptureSourceCocoa::settingsDidChange()
+{
+    m_currentSettings = std::nullopt;
+    RealtimeMediaSource::settingsDidChange();
+}
+
+void DisplayCaptureSourceCocoa::startProducingData()
+{
+#if PLATFORM(IOS)
+    RealtimeMediaSourceCenter::singleton().videoFactory().setActiveSource(*this);
+#endif
+
+    m_startTime = monotonicallyIncreasingTime();
+    m_timer.startRepeating(1_ms * lround(1000 / frameRate()));
+}
+
+void DisplayCaptureSourceCocoa::stopProducingData()
+{
+    m_timer.stop();
+    m_elapsedTime += monotonicallyIncreasingTime() - m_startTime;
+    m_startTime = NAN;
+}
+
+double DisplayCaptureSourceCocoa::elapsedTime()
+{
+    if (std::isnan(m_startTime))
+        return m_elapsedTime;
+
+    return m_elapsedTime + (monotonicallyIncreasingTime() - m_startTime);
+}
+
+bool DisplayCaptureSourceCocoa::applyFrameRate(double rate)
+{
+    if (m_timer.isActive())
+        m_timer.startRepeating(1_ms * lround(1000 / rate));
+
+    return true;
+}
+
+void DisplayCaptureSourceCocoa::emitFrame()
+{
+    if (muted())
+        return;
+
+    generateFrame();
+}
+
+} // namespace WebCore
+
+#endif // ENABLE(MEDIA_STREAM)
diff --git a/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h b/Source/WebCore/platform/mediastream/mac/DisplayCaptureSourceCocoa.h
new file mode 100644 (file)
index 0000000..2cec9dc
--- /dev/null
@@ -0,0 +1,81 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. ``AS IS'' AND ANY
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL APPLE INC. OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#pragma once
+
+#if ENABLE(MEDIA_STREAM)
+
+#include "CaptureDevice.h"
+#include "RealtimeMediaSource.h"
+#include <wtf/RefCounted.h>
+#include <wtf/RefPtr.h>
+#include <wtf/RunLoop.h>
+#include <wtf/text/WTFString.h>
+
+namespace WTF {
+class MediaTime;
+}
+
+namespace WebCore {
+
+class CaptureDeviceInfo;
+
+class DisplayCaptureSourceCocoa : public RealtimeMediaSource {
+public:
+
+protected:
+    DisplayCaptureSourceCocoa(const String& name);
+    virtual ~DisplayCaptureSourceCocoa();
+
+    virtual void generateFrame() = 0;
+    void startProducingData() override;
+    void stopProducingData() override;
+
+    double elapsedTime();
+    bool applyFrameRate(double) override;
+
+private:
+
+    bool isCaptureSource() const final { return true; }
+
+    const RealtimeMediaSourceCapabilities& capabilities() const final;
+    const RealtimeMediaSourceSettings& settings() const final;
+    void settingsDidChange() final;
+
+    void emitFrame();
+
+    mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
+    mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings;
+    RealtimeMediaSourceSupportedConstraints m_supportedConstraints;
+
+    double m_startTime { NAN };
+    double m_elapsedTime { 0 };
+
+    RunLoop::Timer<DisplayCaptureSourceCocoa> m_timer;
+};
+
+} // namespace WebCore
+
+#endif // ENABLE(MEDIA_STREAM)
index 5ea7a932c9c929399d7a0b877b56f25ed136bfb2..735e2639984d1cfcbdc57191526a99cd43ea2c56 100644 (file)
@@ -38,6 +38,7 @@
 #include "AVVideoCaptureSource.h"
 #include "CoreAudioCaptureDeviceManager.h"
 #include "CoreAudioCaptureSource.h"
+#include "DisplayCaptureManagerCocoa.h"
 #include "Logging.h"
 #include "MediaStreamPrivate.h"
 #include <wtf/MainThread.h>
@@ -53,13 +54,12 @@ public:
         case CaptureDevice::DeviceType::Camera:
             return AVVideoCaptureSource::create(device.persistentId(), constraints);
             break;
-        
-        case CaptureDevice::DeviceType::Microphone:
         case CaptureDevice::DeviceType::Screen:
         case CaptureDevice::DeviceType::Application:
         case CaptureDevice::DeviceType::Window:
         case CaptureDevice::DeviceType::Browser:
-        case CaptureDevice::DeviceType::Unknown:        
+        case CaptureDevice::DeviceType::Microphone:
+        case CaptureDevice::DeviceType::Unknown:
             ASSERT_NOT_REACHED();
             break;
         }
@@ -132,6 +132,11 @@ CaptureDeviceManager& RealtimeMediaSourceCenterMac::videoCaptureDeviceManager()
     return AVCaptureDeviceManager::singleton();
 }
 
+CaptureDeviceManager& RealtimeMediaSourceCenterMac::displayCaptureDeviceManager()
+{
+    return DisplayCaptureManagerCocoa::singleton();
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM)
index e4c00feb927aca8c131bcfe27ee4db6eeea4afe1..c754f4c939a0a272443511581d4516ae0ed4a60d 100644 (file)
@@ -60,6 +60,7 @@ private:
 
     CaptureDeviceManager& audioCaptureDeviceManager() final;
     CaptureDeviceManager& videoCaptureDeviceManager() final;
+    CaptureDeviceManager& displayCaptureDeviceManager() final;
 
     RealtimeMediaSource::AudioCaptureFactory* m_audioFactoryOverride { nullptr };
 };
index 3435bd82e1f8bb67c3afecf3bb38b645f3659de3..b72a1b61b48e365d12907cd4dda9446f6595dd82 100644 (file)
@@ -61,6 +61,9 @@ static const HashMap<String, MockDeviceInfo>& deviceMap()
 
             { "239c24b2-2b15-11e3-8224-0800200c9a66", CaptureDevice::DeviceType::Camera, "Mock video device 1", MockRealtimeMediaSource::MockDevice::Camera1 },
             { "239c24b3-2b15-11e3-8224-0800200c9a66", CaptureDevice::DeviceType::Camera, "Mock video device 2", MockRealtimeMediaSource::MockDevice::Camera2 },
+
+            { "SCREEN-1", CaptureDevice::DeviceType::Screen, "Mock screen device 1", MockRealtimeMediaSource::MockDevice::Screen1 },
+            { "SCREEN-2", CaptureDevice::DeviceType::Screen, "Mock screen device 2", MockRealtimeMediaSource::MockDevice::Screen2 },
         };
 
         HashMap<String, MockDeviceInfo> map;
@@ -123,6 +126,25 @@ Vector<CaptureDevice>& MockRealtimeMediaSource::videoDevices()
     return info;
 }
 
+Vector<CaptureDevice>& MockRealtimeMediaSource::displayDevices()
+{
+    static auto devices = makeNeverDestroyed([] {
+        Vector<CaptureDevice> vector;
+
+        auto captureDevice = captureDeviceWithPersistentID(CaptureDevice::DeviceType::Screen, ASCIILiteral("SCREEN-1"));
+        ASSERT(captureDevice);
+        vector.append(WTFMove(captureDevice.value()));
+
+        captureDevice = captureDeviceWithPersistentID(CaptureDevice::DeviceType::Screen, ASCIILiteral("SCREEN-2"));
+        ASSERT(captureDevice);
+        vector.append(WTFMove(captureDevice.value()));
+
+        return vector;
+    }());
+
+    return devices;
+}
+
 MockRealtimeMediaSource::MockRealtimeMediaSource(const String& id, RealtimeMediaSource::Type type, const String& name)
     : RealtimeMediaSource(id, type, name)
 {
index 49de8b8e7d22ea80008f9e9d35d75ba9656ea43a..a1d3e9b61b6113365a784300283b94a5e11fa75f 100644 (file)
@@ -44,10 +44,11 @@ public:
 
     static Vector<CaptureDevice>& audioDevices();
     static Vector<CaptureDevice>& videoDevices();
+    static Vector<CaptureDevice>& displayDevices();
 
     static std::optional<CaptureDevice> captureDeviceWithPersistentID(CaptureDevice::DeviceType, const String&);
 
-    enum class MockDevice { Invalid, Microphone1, Microphone2, Camera1, Camera2 };
+    enum class MockDevice { Invalid, Microphone1, Microphone2, Camera1, Camera2, Screen1, Screen2 };
 
 protected:
     MockRealtimeMediaSource(const String& id, Type, const String& name);
index 4efdf72cfacac8aeef7fec55b12790080d830687..77a0f8edd3cb7f6ae233b750301215368200701c 100644 (file)
 
 #if ENABLE(MEDIA_STREAM)
 
-#include "CaptureDevice.h"
 #include "Logging.h"
-#include "MediaStream.h"
-#include "MediaStreamPrivate.h"
-#include "MediaStreamTrack.h"
 #include "MockRealtimeAudioSource.h"
-#include "MockRealtimeMediaSource.h"
 #include "MockRealtimeVideoSource.h"
-#include "RealtimeMediaSource.h"
-#include "RealtimeMediaSourceCapabilities.h"
 #include <wtf/NeverDestroyed.h>
 
 namespace WebCore {
index 047baafc4f808fa5f1edfaf1dd4f4b1f0d54fce0..0f92f9d43f989ba6e8c3aa8f35c404eb8b429f3f 100644 (file)
@@ -51,6 +51,7 @@ private:
 
     CaptureDeviceManager& audioCaptureDeviceManager() final { return m_audioCaptureDeviceManager; }
     CaptureDeviceManager& videoCaptureDeviceManager() final { return m_videoCaptureDeviceManager; }
+    CaptureDeviceManager& displayCaptureDeviceManager() final { return m_displayCaptureDeviceManager; }
 
     static std::optional<CaptureDevice> captureDeviceWithPersistentID(CaptureDevice::DeviceType, const String&);
 
@@ -64,9 +65,15 @@ private:
         const Vector<CaptureDevice>& captureDevices() final { return MockRealtimeMediaSource::videoDevices(); }
         std::optional<CaptureDevice> captureDeviceWithPersistentID(CaptureDevice::DeviceType type, const String& id) final { return MockRealtimeMediaSource::captureDeviceWithPersistentID(type, id); }
     };
+    class MockDisplayCaptureDeviceManager final : public CaptureDeviceManager {
+    private:
+        const Vector<CaptureDevice>& captureDevices() final { return MockRealtimeMediaSource::displayDevices(); }
+        std::optional<CaptureDevice> captureDeviceWithPersistentID(CaptureDevice::DeviceType type, const String& id) final { return MockRealtimeMediaSource::captureDeviceWithPersistentID(type, id); }
+    };
 
     MockAudioCaptureDeviceManager m_audioCaptureDeviceManager;
     MockVideoCaptureDeviceManager m_videoCaptureDeviceManager;
+    MockDisplayCaptureDeviceManager m_displayCaptureDeviceManager;
 };
 
 }
index dba5a1081da9deb219b304d15545d8bc4572aadf..5fdc3f52dd819f494908cabbccee7f1d5c073e98 100644 (file)
@@ -58,10 +58,9 @@ public:
 
         switch (device.type()) {
         case CaptureDevice::DeviceType::Camera:
+        case CaptureDevice::DeviceType::Screen:
             return MockRealtimeVideoSource::create(device.persistentId(), device.label(), constraints);
             break;
-
-        case CaptureDevice::DeviceType::Screen:
         case CaptureDevice::DeviceType::Application:
         case CaptureDevice::DeviceType::Window:
         case CaptureDevice::DeviceType::Browser:
@@ -119,6 +118,12 @@ MockRealtimeVideoSource::MockRealtimeVideoSource(const String& deviceID, const S
         setFrameRate(15);
         setFacingMode(RealtimeMediaSourceSettings::Environment);
         break;
+    case MockDevice::Screen1:
+        setFrameRate(30);
+        break;
+    case MockDevice::Screen2:
+        setFrameRate(10);
+        break;
     case MockDevice::Microphone1:
     case MockDevice::Microphone2:
     case MockDevice::Invalid:
@@ -170,7 +175,12 @@ double MockRealtimeVideoSource::elapsedTime()
 
 void MockRealtimeVideoSource::updateSettings(RealtimeMediaSourceSettings& settings)
 {
-    settings.setFacingMode(facingMode());
+    if (mockCamera())
+        settings.setFacingMode(facingMode());
+    else {
+        settings.setDisplaySurface(RealtimeMediaSourceSettings::DisplaySurfaceType::Monitor);
+        settings.setLogicalSurface(true);
+    }
     settings.setFrameRate(frameRate());
     IntSize size = this->size();
     settings.setWidth(size.width());
@@ -181,15 +191,21 @@ void MockRealtimeVideoSource::updateSettings(RealtimeMediaSourceSettings& settin
 
 void MockRealtimeVideoSource::initializeCapabilities(RealtimeMediaSourceCapabilities& capabilities)
 {
-    if (device() == MockDevice::Camera1)
-        capabilities.addFacingMode(RealtimeMediaSourceSettings::User);
-    else
-        capabilities.addFacingMode(RealtimeMediaSourceSettings::Environment);
-
-    capabilities.setWidth(CapabilityValueOrRange(320, 1920));
-    capabilities.setHeight(CapabilityValueOrRange(240, 1080));
-    capabilities.setFrameRate(CapabilityValueOrRange(15.0, 60.0));
-    capabilities.setAspectRatio(CapabilityValueOrRange(4 / 3.0, 16 / 9.0));
+    if (mockCamera()) {
+        if (device() == MockDevice::Camera1)
+            capabilities.addFacingMode(RealtimeMediaSourceSettings::User);
+        else
+            capabilities.addFacingMode(RealtimeMediaSourceSettings::Environment);
+
+        capabilities.setWidth(CapabilityValueOrRange(320, 1920));
+        capabilities.setHeight(CapabilityValueOrRange(240, 1080));
+        capabilities.setFrameRate(CapabilityValueOrRange(15.0, 60.0));
+        capabilities.setAspectRatio(CapabilityValueOrRange(4 / 3.0, 16 / 9.0));
+    } else {
+        capabilities.setWidth(CapabilityValueOrRange(72, 2880));
+        capabilities.setHeight(CapabilityValueOrRange(45, 1800));
+        capabilities.setFrameRate(CapabilityValueOrRange(.01, 60.0));
+    }
 }
 
 void MockRealtimeVideoSource::initializeSupportedConstraints(RealtimeMediaSourceSupportedConstraints& supportedConstraints)
@@ -198,7 +214,8 @@ void MockRealtimeVideoSource::initializeSupportedConstraints(RealtimeMediaSource
     supportedConstraints.setSupportsHeight(true);
     supportedConstraints.setSupportsAspectRatio(true);
     supportedConstraints.setSupportsFrameRate(true);
-    supportedConstraints.setSupportsFacingMode(true);
+    if (mockCamera())
+        supportedConstraints.setSupportsFacingMode(true);
 }
 
 bool MockRealtimeVideoSource::applyFrameRate(double rate)
@@ -352,27 +369,32 @@ void MockRealtimeVideoSource::drawText(GraphicsContext& context)
     statsLocation.move(0, m_statsFontSize);
     context.drawText(statsFont, TextRun((StringView(string))), statsLocation);
 
-    const char* camera;
-    switch (facingMode()) {
-    case RealtimeMediaSourceSettings::User:
-        camera = "User facing";
-        break;
-    case RealtimeMediaSourceSettings::Environment:
-        camera = "Environment facing";
-        break;
-    case RealtimeMediaSourceSettings::Left:
-        camera = "Left facing";
-        break;
-    case RealtimeMediaSourceSettings::Right:
-        camera = "Right facing";
-        break;
-    case RealtimeMediaSourceSettings::Unknown:
-        camera = "Unknown";
-        break;
+    if (mockCamera()) {
+        const char* camera;
+        switch (facingMode()) {
+        case RealtimeMediaSourceSettings::User:
+            camera = "User facing";
+            break;
+        case RealtimeMediaSourceSettings::Environment:
+            camera = "Environment facing";
+            break;
+        case RealtimeMediaSourceSettings::Left:
+            camera = "Left facing";
+            break;
+        case RealtimeMediaSourceSettings::Right:
+            camera = "Right facing";
+            break;
+        case RealtimeMediaSourceSettings::Unknown:
+            camera = "Unknown";
+            break;
+        }
+        string = String::format("Camera: %s", camera);
+        statsLocation.move(0, m_statsFontSize);
+        context.drawText(statsFont, TextRun((StringView(string))), statsLocation);
+    } else {
+        statsLocation.move(0, m_statsFontSize);
+        context.drawText(statsFont, TextRun((StringView(device() == MockDevice::Screen1 ? "Screen 1" : "Screen 2"))), statsLocation);
     }
-    string = String::format("Camera: %s", camera);
-    statsLocation.move(0, m_statsFontSize);
-    context.drawText(statsFont, TextRun((StringView(string))), statsLocation);
 
     FloatPoint bipBopLocation(size.width() * .6, size.height() * .6);
     unsigned frameMod = m_frameNumber % 60;
@@ -418,6 +440,12 @@ void MockRealtimeVideoSource::generateFrame()
     case MockDevice::Camera2:
         fillColor = Color::darkGray;
         break;
+    case MockDevice::Screen1:
+        fillColor = Color::lightGray;
+        break;
+    case MockDevice::Screen2:
+        fillColor = Color::yellow;
+        break;
     case MockDevice::Microphone1:
     case MockDevice::Microphone2:
     case MockDevice::Invalid:
index a0a5e6f3899387699a7475d85d4e0e32fb600559..94c8be9baf4c5c60adc6a218806af6fdb30edcfc 100644 (file)
@@ -83,6 +83,9 @@ private:
 
     void delaySamples(float) override;
 
+    bool mockCamera() const { return device() == MockDevice::Camera1 || device() == MockDevice::Camera2; }
+    bool mockScreen() const { return device() == MockDevice::Screen1 || device() == MockDevice::Screen2; }
+
     float m_baseFontSize { 0 };
     float m_bipBopFontSize { 0 };
     float m_statsFontSize { 0 };
index 3c154bae973b05837de3337ce3502acec45f95d9..1e3886ec3e267a645e1dd548e2a37890df574935 100644 (file)
@@ -1,3 +1,37 @@
+2018-01-04  Eric Carlson  <eric.carlson@apple.com>
+
+        [MediaStream] Add Mock screen capture source
+        https://bugs.webkit.org/show_bug.cgi?id=181291
+        <rdar://problem/36298164>
+
+        Reviewed by Dean Jackson.
+
+        * Shared/WebCoreArgumentCoders.cpp:
+        (IPC::ArgumentCoder<MediaConstraints>::decode):
+        (IPC::ArgumentCoder<CaptureDevice>::encode): Deleted, moved to CaptureDevice.h
+        (IPC::ArgumentCoder<CaptureDevice>::decode): Ditto.
+        * Shared/WebCoreArgumentCoders.h:
+
+        * UIProcess/API/Cocoa/WKWebViewPrivate.h: Add _WKCaptureDeviceDisplay.
+        * UIProcess/Cocoa/UIDelegate.mm:
+        (WebKit::requestUserMediaAuthorizationForDevices): Deal with display capture.
+        (WebKit::UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest): Ditto.
+
+        * UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
+        (WebKit::UserMediaPermissionRequestManagerProxy::userMediaAccessWasDenied): requiresAudio -> requiresAudioCapture.
+        (WebKit::UserMediaPermissionRequestManagerProxy::searchForGrantedRequest const): Never reuse
+        a previously granted display capture request.
+
+        * UIProcess/UserMediaPermissionRequestProxy.cpp:
+        (WebKit::UserMediaPermissionRequestProxy::allow): Search the eligible devices instead of asking
+        the source center to find devices.
+        * UIProcess/UserMediaPermissionRequestProxy.h:
+        (WebKit::UserMediaPermissionRequestProxy::requiresAudioCapture const): Renamed.
+        (WebKit::UserMediaPermissionRequestProxy::requiresVideoCapture const): Ditto.
+        (WebKit::UserMediaPermissionRequestProxy::requiresDisplayCapture const): New.
+        (WebKit::UserMediaPermissionRequestProxy::requiresAudio const): Deleted.
+        (WebKit::UserMediaPermissionRequestProxy::requiresVideo const): Deleted.
+
 2018-01-04  Youenn Fablet  <youenn@apple.com>
 
         FetchResponse should set its internal response text encoding name
index 49ce56aa55a212e9d63dd96690146c52a514e8e9..b565ecbed7ab5aba9bcc762968c8533b67cafff5 100644 (file)
@@ -2598,47 +2598,6 @@ bool ArgumentCoder<MediaConstraints>::decode(Decoder& decoder, WebCore::MediaCon
         && decoder.decode(constraints.deviceIDHashSalt)
         && decoder.decode(constraints.isValid);
 }
-
-void ArgumentCoder<CaptureDevice>::encode(Encoder& encoder, const WebCore::CaptureDevice& device)
-{
-    encoder << device.persistentId();
-    encoder << device.label();
-    encoder << device.groupId();
-    encoder << device.enabled();
-    encoder.encodeEnum(device.type());
-}
-
-std::optional<CaptureDevice> ArgumentCoder<CaptureDevice>::decode(Decoder& decoder)
-{
-    std::optional<String> persistentId;
-    decoder >> persistentId;
-    if (!persistentId)
-        return std::nullopt;
-
-    std::optional<String> label;
-    decoder >> label;
-    if (!label)
-        return std::nullopt;
-
-    std::optional<String> groupId;
-    decoder >> groupId;
-    if (!groupId)
-        return std::nullopt;
-
-    std::optional<bool> enabled;
-    decoder >> enabled;
-    if (!enabled)
-        return std::nullopt;
-
-    std::optional<CaptureDevice::DeviceType> type;
-    decoder >> type;
-    if (!type)
-        return std::nullopt;
-
-    std::optional<CaptureDevice> device = {{ WTFMove(*persistentId), WTFMove(*type), WTFMove(*label), WTFMove(*groupId) }};
-    device->setEnabled(*enabled);
-    return device;
-}
 #endif
 
 #if ENABLE(INDEXED_DATABASE)
index a2cd09bb1cd5305675f8f8a229c6d4c9a7063ded..8c69d1fec3cfc5d6b0fd4d1ec9260515062083b8 100644 (file)
@@ -28,7 +28,6 @@
 #include "ArgumentCoders.h"
 #include <WebCore/AutoplayEvent.h>
 #include <WebCore/CacheStorageConnection.h>
-#include <WebCore/CaptureDevice.h>
 #include <WebCore/ColorSpace.h>
 #include <WebCore/DiagnosticLoggingClient.h>
 #include <WebCore/FrameLoaderTypes.h>
@@ -152,7 +151,6 @@ class MediaSessionMetadata;
 #endif
 
 #if ENABLE(MEDIA_STREAM)
-class CaptureDevice;
 struct MediaConstraints;
 #endif
 
@@ -648,11 +646,6 @@ template<> struct ArgumentCoder<WebCore::MediaConstraints> {
     static void encode(Encoder&, const WebCore::MediaConstraints&);
     static bool decode(Decoder&, WebCore::MediaConstraints&);
 };
-
-template<> struct ArgumentCoder<WebCore::CaptureDevice> {
-    static void encode(Encoder&, const WebCore::CaptureDevice&);
-    static std::optional<WebCore::CaptureDevice> decode(Decoder&);
-};
 #endif
 
 #if ENABLE(INDEXED_DATABASE)
@@ -770,14 +763,6 @@ template<> struct EnumTraits<WebCore::IndexedDB::GetAllType> {
 #endif
 
 #if ENABLE(MEDIA_STREAM)
-template<> struct EnumTraits<WebCore::CaptureDevice::DeviceType> {
-    using values = EnumValues<
-        WebCore::CaptureDevice::DeviceType,
-        WebCore::CaptureDevice::DeviceType::Unknown,
-        WebCore::CaptureDevice::DeviceType::Microphone,
-        WebCore::CaptureDevice::DeviceType::Camera
-    >;
-};
 template<> struct EnumTraits<WebCore::RealtimeMediaSource::Type> {
     using values = EnumValues<
         WebCore::RealtimeMediaSource::Type,
index c0b40b042f942b826b39d8dc93a491d9dc58914b..a5a70878b8391c67332c3edd9110e3725f70c4b4 100644 (file)
@@ -59,6 +59,7 @@ typedef NS_OPTIONS(NSInteger, _WKMediaMutedState) {
 typedef NS_OPTIONS(NSUInteger, _WKCaptureDevices) {
     _WKCaptureDeviceMicrophone = 1 << 0,
     _WKCaptureDeviceCamera = 1 << 1,
+    _WKCaptureDeviceDisplay = 1 << 2,
 } WK_API_AVAILABLE(macosx(10.13), ios(11.0));
 
 #if TARGET_OS_IPHONE
index a285190c52df4b7022c0762c583735546d3975fc..7e9fa0cbab466a84ecbced2177a10c7aeb9b46d5 100644 (file)
@@ -793,8 +793,8 @@ static void requestUserMediaAuthorizationForDevices(const WebFrameProxy& frame,
             protectedRequest->deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied);
             return;
         }
-        const String& videoDeviceUID = protectedRequest->requiresVideo() ? protectedRequest->videoDeviceUIDs().first() : String();
-        const String& audioDeviceUID = protectedRequest->requiresAudio() ? protectedRequest->audioDeviceUIDs().first() : String();
+        const String& videoDeviceUID = (protectedRequest->requiresVideoCapture() || protectedRequest->requiresDisplayCapture()) ? protectedRequest->videoDeviceUIDs().first() : String();
+        const String& audioDeviceUID = protectedRequest->requiresAudioCapture() ? protectedRequest->audioDeviceUIDs().first() : String();
         protectedRequest->allow(audioDeviceUID, videoDeviceUID);
     });
 
@@ -803,10 +803,14 @@ static void requestUserMediaAuthorizationForDevices(const WebFrameProxy& frame,
     WebCore::URL mainFrameURL(WebCore::URL(), mainFrame->url());
 
     _WKCaptureDevices devices = 0;
-    if (request.requiresAudio())
+    if (request.requiresAudioCapture())
         devices |= _WKCaptureDeviceMicrophone;
-    if (request.requiresVideo())
+    if (request.requiresVideoCapture())
         devices |= _WKCaptureDeviceCamera;
+    if (request.requiresDisplayCapture()) {
+        devices |= _WKCaptureDeviceDisplay;
+        ASSERT(!(devices & _WKCaptureDeviceCamera));
+    }
 
     auto protectedWebView = RetainPtr<WKWebView>(&webView);
     [delegate _webView:protectedWebView.get() requestUserMediaAuthorizationForDevices:devices url:requestFrameURL mainFrameURL:mainFrameURL decisionHandler:decisionHandler.get()];
@@ -820,9 +824,10 @@ bool UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest(WebPageProx
         return true;
     }
 
-    bool requiresAudio = request.requiresAudio();
-    bool requiresVideo = request.requiresVideo();
-    if (!requiresAudio && !requiresVideo) {
+    bool requiresAudioCapture = request.requiresAudioCapture();
+    bool requiresVideoCapture = request.requiresVideoCapture();
+    bool requiresDisplayCapture = request.requiresDisplayCapture();
+    if (!requiresAudioCapture && !requiresVideoCapture && !requiresDisplayCapture) {
         request.deny(UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints);
         return true;
     }
@@ -830,7 +835,7 @@ bool UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest(WebPageProx
 #if PLATFORM(IOS)
     auto requestCameraAuthorization = BlockPtr<void()>::fromCallable([this, &frame, protectedRequest = makeRef(request), webView = RetainPtr<WKWebView>(m_uiDelegate.m_webView)]() {
 
-        if (!protectedRequest->requiresVideo()) {
+        if (!protectedRequest->requiresVideoCapture()) {
             requestUserMediaAuthorizationForDevices(frame, protectedRequest, (id <WKUIDelegatePrivate>)m_uiDelegate.m_delegate.get(), *webView.get());
             return;
         }
@@ -857,7 +862,7 @@ bool UIDelegate::UIClient::decidePolicyForUserMediaPermissionRequest(WebPageProx
         }
     });
 
-    if (requiresAudio) {
+    if (requiresAudioCapture) {
         AVAuthorizationStatus microphoneAuthorizationStatus = [getAVCaptureDeviceClass() authorizationStatusForMediaType:getAVMediaTypeAudio()];
         switch (microphoneAuthorizationStatus) {
         case AVAuthorizationStatusAuthorized:
index 3751d59fb0225f6c0c5a73cb842ca4bdd02edefb..041d7b81a1bfabca54f0051c675727334868e5e3 100644 (file)
@@ -122,7 +122,7 @@ void UserMediaPermissionRequestManagerProxy::userMediaAccessWasDenied(uint64_t u
         return;
 
     if (reason == UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::PermissionDenied)
-        m_deniedRequests.append(DeniedRequest { request->mainFrameID(), request->userMediaDocumentSecurityOrigin(), request->topLevelDocumentSecurityOrigin(), request->requiresAudio(), request->requiresVideo() });
+        m_deniedRequests.append(DeniedRequest { request->mainFrameID(), request->userMediaDocumentSecurityOrigin(), request->topLevelDocumentSecurityOrigin(), request->requiresAudioCapture(), request->requiresVideoCapture() });
 
     denyRequest(userMediaID, reason, emptyString());
 }
@@ -178,6 +178,8 @@ const UserMediaPermissionRequestProxy* UserMediaPermissionRequestManagerProxy::s
     bool checkForAudio = needsAudio;
     bool checkForVideo = needsVideo;
     for (const auto& grantedRequest : m_grantedRequests) {
+        if (grantedRequest->requiresDisplayCapture())
+            continue;
         if (!grantedRequest->userMediaDocumentSecurityOrigin().isSameSchemeHostPort(userMediaDocumentOrigin))
             continue;
         if (!grantedRequest->topLevelDocumentSecurityOrigin().isSameSchemeHostPort(topLevelDocumentOrigin))
@@ -185,10 +187,10 @@ const UserMediaPermissionRequestProxy* UserMediaPermissionRequestManagerProxy::s
         if (grantedRequest->frameID() != frameID)
             continue;
 
-        if (grantedRequest->requiresVideo())
+        if (grantedRequest->requiresVideoCapture())
             checkForVideo = false;
 
-        if (grantedRequest->requiresAudio())
+        if (grantedRequest->requiresAudioCapture())
             checkForAudio = false;
 
         if (checkForVideo || checkForAudio)
index 3e5a9572ac31049d810f24bbf8475dfa3bc1023c..674044ae3b789140be80236d6fb696b5e0234753 100644 (file)
@@ -52,23 +52,30 @@ void UserMediaPermissionRequestProxy::allow(const String& audioDeviceUID, const
         return;
 
 #if ENABLE(MEDIA_STREAM)
-
-    auto& sourceCenter = RealtimeMediaSourceCenter::singleton();
     CaptureDevice audioDevice;
     if (!audioDeviceUID.isEmpty()) {
-        auto device = sourceCenter.captureDeviceWithPersistentID(WebCore::CaptureDevice::DeviceType::Microphone, audioDeviceUID);
-        ASSERT(device && device.value().enabled());
-        if (device)
-            audioDevice = device.value();
+        size_t index = m_eligibleAudioDevices.findMatching([&](const auto& device) {
+            return device.persistentId() == audioDeviceUID;
+        });
+        ASSERT(index != notFound);
+
+        if (index != notFound)
+            audioDevice = m_eligibleAudioDevices[index];
+
+        ASSERT(audioDevice.enabled());
     }
 
     CaptureDevice videoDevice;
     if (!videoDeviceUID.isEmpty()) {
-        auto device = sourceCenter.captureDeviceWithPersistentID(WebCore::CaptureDevice::DeviceType::Camera, videoDeviceUID);
+        size_t index = m_eligibleVideoDevices.findMatching([&](const auto& device) {
+            return device.persistentId() == videoDeviceUID;
+        });
+        ASSERT(index != notFound);
+
+        if (index != notFound)
+            videoDevice = m_eligibleVideoDevices[index];
 
-        ASSERT(device && device.value().enabled());
-        if (device)
-            videoDevice = device.value();
+        ASSERT(videoDevice.enabled());
     }
 
     m_manager->userMediaAccessWasGranted(m_userMediaID, WTFMove(audioDevice), WTFMove(videoDevice));
index 987d26a55c5a6bb4ff0350057378aa6d43d149cf..0a1f18fc9fe0436cd8bc92d0e2886e971e1deb5c 100644 (file)
@@ -49,8 +49,9 @@ public:
 
     void invalidate();
 
-    bool requiresAudio() const { return m_eligibleAudioDevices.size(); }
-    bool requiresVideo() const { return m_eligibleVideoDevices.size(); }
+    bool requiresAudioCapture() const { return m_eligibleAudioDevices.size(); }
+    bool requiresVideoCapture() const { return !requiresDisplayCapture() && m_eligibleVideoDevices.size(); }
+    bool requiresDisplayCapture() const { return m_request.type == WebCore::MediaStreamRequest::Type::DisplayMedia && m_eligibleVideoDevices.size(); }
 
     Vector<String> videoDeviceUIDs() const;
     Vector<String> audioDeviceUIDs() const;
index d19e6d314f76fa2dbb6fce05ba7b93583ff64da9..06c1b027b3a7441f4180e8f566d59fc3d6c4a797 100644 (file)
@@ -1,3 +1,15 @@
+2018-01-04  Eric Carlson  <eric.carlson@apple.com>
+
+        [MediaStream] Add Mock screen capture source
+        https://bugs.webkit.org/show_bug.cgi?id=181291
+        <rdar://problem/36298164>
+
+        Reviewed by Dean Jackson.
+
+        * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj: Add new test.
+        * TestWebKitAPI/Tests/WebKitCocoa/GetDisplayMedia.mm:
+        * TestWebKitAPI/Tests/WebKit/getDisplayMedia.html:
+
 2018-01-04  Lucas Forschler  <lforschler@apple.com>
 
         <rdar://problem/36300930> Change proxy server setting on master config to reach s3 servers
index 724789034481506e51726819ac51cc27c4c7438a..088525c9001630ae5e0b73ce6d3f2dd30c776ffc 100644 (file)
@@ -30,6 +30,8 @@
                0799C34B1EBA3301003B7532 /* disableGetUserMedia.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 0799C34A1EBA32F4003B7532 /* disableGetUserMedia.html */; };
                07C046CA1E4262A8007201E7 /* CARingBuffer.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 07C046C91E42573E007201E7 /* CARingBuffer.cpp */; };
                07CE1CF31F06A7E000BF89F5 /* GetUserMediaNavigation.mm in Sources */ = {isa = PBXBuildFile; fileRef = 07CE1CF21F06A7E000BF89F5 /* GetUserMediaNavigation.mm */; };
+               07E1F6A21FFC44FA0096C7EC /* getDisplayMedia.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 07E1F6A11FFC44F90096C7EC /* getDisplayMedia.html */; };
+               07E1F6A31FFC4B760096C7EC /* GetDisplayMedia.mm in Sources */ = {isa = PBXBuildFile; fileRef = 07E1F6A01FFC3A080096C7EC /* GetDisplayMedia.mm */; };
                07E499911F9E56DF002F1EF3 /* GetUserMediaReprompt.mm in Sources */ = {isa = PBXBuildFile; fileRef = 07E499901F9E56A1002F1EF3 /* GetUserMediaReprompt.mm */; };
                0F139E771A423A5B00F590F5 /* WeakObjCPtr.mm in Sources */ = {isa = PBXBuildFile; fileRef = 0F139E751A423A5300F590F5 /* WeakObjCPtr.mm */; };
                0F139E781A423A6B00F590F5 /* PlatformUtilitiesCocoa.mm in Sources */ = {isa = PBXBuildFile; fileRef = 0F139E721A423A2B00F590F5 /* PlatformUtilitiesCocoa.mm */; };
                                26F52EAF18288C230023D412 /* geolocationGetCurrentPositionWithHighAccuracy.html in Copy Resources */,
                                26F52EB218288F240023D412 /* geolocationWatchPosition.html in Copy Resources */,
                                26F52EB318288F240023D412 /* geolocationWatchPositionWithHighAccuracy.html in Copy Resources */,
+                               07E1F6A21FFC44FA0096C7EC /* getDisplayMedia.html in Copy Resources */,
                                074994421EA5034B000DA44E /* getUserMedia.html in Copy Resources */,
                                F46A095B1ED8A6E600D4AA55 /* gif-and-file-input.html in Copy Resources */,
                                9B4F8FA7159D52DD002D9F94 /* HTMLCollectionNamedItem.html in Copy Resources */,
                0799C34A1EBA32F4003B7532 /* disableGetUserMedia.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = disableGetUserMedia.html; sourceTree = "<group>"; };
                07C046C91E42573E007201E7 /* CARingBuffer.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = CARingBuffer.cpp; sourceTree = "<group>"; };
                07CE1CF21F06A7E000BF89F5 /* GetUserMediaNavigation.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = GetUserMediaNavigation.mm; sourceTree = "<group>"; };
+               07E1F6A01FFC3A080096C7EC /* GetDisplayMedia.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = GetDisplayMedia.mm; sourceTree = "<group>"; };
+               07E1F6A11FFC44F90096C7EC /* getDisplayMedia.html */ = {isa = PBXFileReference; lastKnownFileType = text.html; name = getDisplayMedia.html; path = ../WebKit/getDisplayMedia.html; sourceTree = "<group>"; };
                07E499901F9E56A1002F1EF3 /* GetUserMediaReprompt.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = GetUserMediaReprompt.mm; sourceTree = "<group>"; };
                07EDEFAC1EB9400C00D43292 /* UserMediaDisabled.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = UserMediaDisabled.mm; sourceTree = "<group>"; };
                0BCD833414857CE400EA2003 /* HashMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HashMap.cpp; sourceTree = "<group>"; };
                                3F1B52681D3D7129008D60C4 /* FullscreenLayoutConstraints.mm */,
                                CDE195B31CFE0ADE0053D256 /* FullscreenTopContentInset.mm */,
                                631EFFF51E7B5E8D00D2EBB8 /* Geolocation.mm */,
+                               07E1F6A01FFC3A080096C7EC /* GetDisplayMedia.mm */,
                                51AF23DE1EF1A3720072F281 /* IconLoadingDelegate.mm */,
                                510477751D298E03009747EB /* IDBDeleteRecovery.mm */,
                                5110FCEF1E01CBAA006F8D0B /* IDBIndexUpgradeToV2.mm */,
                                3FBD1B491D39D1DB00E6D6FA /* FullscreenLayoutConstraints.html */,
                                CDE195B21CFE0ADE0053D256 /* FullscreenTopContentInset.html */,
                                636353A61E9861940009F8AF /* GeolocationGetCurrentPositionResult.html */,
+                               07E1F6A11FFC44F90096C7EC /* getDisplayMedia.html */,
                                F47D30ED1ED28A6C000482E1 /* gif-and-file-input.html */,
                                510477761D298E57009747EB /* IDBDeleteRecovery.html */,
                                5104776F1D298D85009747EB /* IDBDeleteRecovery.sqlite3 */,
                                7CCE7EF81A411AE600447C4C /* Geolocation.cpp in Sources */,
                                631EFFF61E7B5E8D00D2EBB8 /* Geolocation.mm in Sources */,
                                7CCE7EE11A411A9A00447C4C /* GetBackingScaleFactor.mm in Sources */,
+                               07E1F6A31FFC4B760096C7EC /* GetDisplayMedia.mm in Sources */,
                                7CCE7EF91A411AE600447C4C /* GetInjectedBundleInitializationUserDataCallback.cpp in Sources */,
                                7CCE7EE21A411A9A00447C4C /* GetPIDAfterAbortedProcessLaunch.cpp in Sources */,
                                07CE1CF31F06A7E000BF89F5 /* GetUserMediaNavigation.mm in Sources */,
diff --git a/Tools/TestWebKitAPI/Tests/WebKit/getDisplayMedia.html b/Tools/TestWebKitAPI/Tests/WebKit/getDisplayMedia.html
new file mode 100644 (file)
index 0000000..f11ba31
--- /dev/null
@@ -0,0 +1,55 @@
+<!DOCTYPE html>
+<html>
+    <head>
+        <script>
+
+            let stream = null;
+
+            function promptForCapture(constraints)
+            {
+                navigator.mediaDevices.getDisplayMedia(constraints)
+                .then((s) => {
+                    stream = s;
+                    video.srcObject = stream;
+                    if (window.webkit)
+                      window.webkit.messageHandlers.testHandler.postMessage('allowed');
+                })
+                .catch((error) => {
+                    if (window.webkit)
+                       window.webkit.messageHandlers.testHandler.postMessage('denied');
+                });
+            }
+
+            function stop(kind)
+            {
+                if (!stream)
+                    return;
+
+                let activeTracks = [];
+                stream.getTracks().forEach(track => {
+                    if (!kind || track.kind == kind)
+                        track.stop();
+                    else
+                        activeTracks.push(track);
+                });
+
+                if (!activeTracks.length) {
+                    stream = null;
+                    video.srcObject = null;
+                }
+            }
+
+            function haveStream()
+            {
+                return stream !== null;
+            }
+        </script>
+    <head>
+
+    <body>
+        <video id="video" controls></video>
+        <p>
+        <button onclick="stop()">Stop</button>
+        </p>
+    </body>
+</html>
diff --git a/Tools/TestWebKitAPI/Tests/WebKitCocoa/GetDisplayMedia.mm b/Tools/TestWebKitAPI/Tests/WebKitCocoa/GetDisplayMedia.mm
new file mode 100644 (file)
index 0000000..2cea279
--- /dev/null
@@ -0,0 +1,171 @@
+/*
+ * Copyright (C) 2017 Apple Inc. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS''
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+ * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
+ * THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#import "config.h"
+
+#if WK_API_ENABLED
+
+#if ENABLE(MEDIA_STREAM)
+
+#import "PlatformUtilities.h"
+#import "Test.h"
+#import "TestWKWebView.h"
+#import <WebKit/WKPreferencesPrivate.h>
+#import <WebKit/WKUIDelegatePrivate.h>
+#import <WebKit/WKWebView.h>
+#import <WebKit/WKWebViewConfiguration.h>
+
+static bool wasPrompted = false;
+static _WKCaptureDevices requestedDevices = 0;
+static bool receivedScriptMessage = false;
+static RetainPtr<WKScriptMessage> lastScriptMessage;
+
+@interface GetDisplayMediaMessageHandler : NSObject <WKScriptMessageHandler>
+@end
+
+@implementation GetDisplayMediaMessageHandler
+
+- (void)userContentController:(WKUserContentController *)userContentController didReceiveScriptMessage:(WKScriptMessage *)message
+{
+    lastScriptMessage = message;
+    receivedScriptMessage = true;
+}
+@end
+
+@interface GetDisplayMediaUIDelegate : NSObject<WKUIDelegate>
+- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForDevices:(_WKCaptureDevices)devices url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorized))decisionHandler;
+- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler;
+@end
+
+@implementation GetDisplayMediaUIDelegate
+- (void)_webView:(WKWebView *)webView requestUserMediaAuthorizationForDevices:(_WKCaptureDevices)devices url:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL decisionHandler:(void (^)(BOOL authorized))decisionHandler
+{
+    wasPrompted = true;
+
+    requestedDevices = devices;
+    BOOL needsMicrophoneAuthorization = !!(requestedDevices & _WKCaptureDeviceMicrophone);
+    BOOL needsCameraAuthorization = !!(requestedDevices & _WKCaptureDeviceCamera);
+    BOOL needsDisplayCaptureAuthorization = !!(requestedDevices & _WKCaptureDeviceDisplay);
+    if (!needsMicrophoneAuthorization && !needsCameraAuthorization && !needsDisplayCaptureAuthorization) {
+        decisionHandler(NO);
+        return;
+    }
+
+    decisionHandler(YES);
+}
+
+- (void)_webView:(WKWebView *)webView checkUserMediaPermissionForURL:(NSURL *)url mainFrameURL:(NSURL *)mainFrameURL frameIdentifier:(NSUInteger)frameIdentifier decisionHandler:(void (^)(NSString *salt, BOOL authorized))decisionHandler
+{
+    decisionHandler(@"0x987654321", YES);
+}
+@end
+
+namespace TestWebKitAPI {
+
+class GetDisplayMediaTest : public testing::Test {
+public:
+    virtual void SetUp()
+    {
+        m_configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+
+        auto handler = adoptNS([[GetDisplayMediaMessageHandler alloc] init]);
+        [[m_configuration userContentController] addScriptMessageHandler:handler.get() name:@"testHandler"];
+
+        auto preferences = [m_configuration preferences];
+        preferences._mediaCaptureRequiresSecureConnection = NO;
+        preferences._mediaDevicesEnabled = YES;
+        preferences._mockCaptureDevicesEnabled = YES;
+        preferences._screenCaptureEnabled = YES;
+
+        m_webView = adoptNS([[TestWKWebView alloc] initWithFrame:NSMakeRect(0, 0, 800, 600) configuration:m_configuration.get()]);
+
+        m_uiDelegate = adoptNS([[GetDisplayMediaUIDelegate alloc] init]);
+        m_webView.get().UIDelegate = m_uiDelegate.get();
+
+        [m_webView synchronouslyLoadTestPageNamed:@"getDisplayMedia"];
+    }
+
+    bool haveStream(bool expected)
+    {
+        int retryCount = 10;
+        while (retryCount--) {
+            auto result = [m_webView stringByEvaluatingJavaScript:@"haveStream()"];
+            if (result.boolValue == expected)
+                return YES;
+
+            TestWebKitAPI::Util::spinRunLoop(10);
+        }
+
+        return NO;
+    }
+
+    void promptForCapture(NSString* constraints, bool shouldSucceed)
+    {
+        [m_webView stringByEvaluatingJavaScript:@"stop()"];
+        EXPECT_TRUE(haveStream(false));
+
+        wasPrompted = false;
+        receivedScriptMessage = false;
+
+        NSString *script = [NSString stringWithFormat:@"promptForCapture(%@)", constraints];
+        [m_webView stringByEvaluatingJavaScript:script];
+
+        TestWebKitAPI::Util::run(&receivedScriptMessage);
+        if (shouldSucceed) {
+            EXPECT_STREQ([(NSString *)[lastScriptMessage body] UTF8String], "allowed");
+            EXPECT_TRUE(haveStream(true));
+            EXPECT_TRUE(wasPrompted);
+        } else {
+            EXPECT_STREQ([(NSString *)[lastScriptMessage body] UTF8String], "denied");
+            EXPECT_TRUE(haveStream(false));
+            EXPECT_FALSE(wasPrompted);
+        }
+    }
+
+    RetainPtr<WKWebViewConfiguration> m_configuration;
+    RetainPtr<GetDisplayMediaUIDelegate> m_uiDelegate;
+    RetainPtr<TestWKWebView> m_webView;
+};
+
+TEST_F(GetDisplayMediaTest, BasicPrompt)
+{
+    promptForCapture(@"{ audio: true, video: true }", true);
+    promptForCapture(@"{ audio: true, video: false }", true);
+    promptForCapture(@"{ audio: false, video: true }", true);
+    promptForCapture(@"{ audio: false, video: false }", false);
+}
+
+TEST_F(GetDisplayMediaTest, Constraints)
+{
+    promptForCapture(@"{ video: {width: 640} }", false);
+    promptForCapture(@"{ video: true, audio: { volume: 0.5 } }", false);
+    promptForCapture(@"{ video: {height: 480}, audio: true }", false);
+}
+
+} // namespace TestWebKitAPI
+
+#endif // ENABLE(MEDIA_STREAM)
+
+#endif // WK_API_ENABLED