[MediaStream] Host should be able to mute screen capture and camera/microphone indepe...
authoreric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Apr 2019 20:44:29 +0000 (20:44 +0000)
committereric.carlson@apple.com <eric.carlson@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 4 Apr 2019 20:44:29 +0000 (20:44 +0000)
https://bugs.webkit.org/show_bug.cgi?id=196555
<rdar://problem/47303865>

Reviewed by Youenn Fablet.

Source/WebCore:

Test: fast/mediastream/media-stream-page-muted.html

* Modules/mediastream/MediaStream.cpp:
(WebCore::MediaStream::MediaStream): Call setCaptureTracksMuted to pass page muted
state to tracks.
(WebCore::MediaStream::startProducingData): Ditto.
(WebCore::MediaStream::setCaptureTracksMuted): New.
* Modules/mediastream/MediaStream.h:

* Modules/mediastream/MediaStreamTrack.cpp:
(WebCore::MediaStreamTrack::MediaStreamTrack): Call setMuted with page muted state.
(WebCore::MediaStreamTrack::setMuted): Set muted according to page state and source type.
(WebCore::MediaStreamTrack::pageMutedStateDidChange): Call setMuted.
(WebCore::MediaStreamTrack::mediaState const): Update for new page state.
* Modules/mediastream/MediaStreamTrack.h:

* page/MediaProducer.h: Split capture muted state into two: camera/microphone and screen.

* page/Page.h:
(WebCore::Page::isMediaCaptureMuted const): Update for state changes.

* platform/mediastream/MediaStreamPrivate.cpp:
(WebCore::MediaStreamPrivate::setCaptureTracksMuted): Deleted.
(WebCore::MediaStreamPrivate::hasCaptureVideoSource const): Deleted.
* platform/mediastream/MediaStreamPrivate.h:

* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::setMuted): Log state.
(WebCore::RealtimeMediaSource::notifyMutedChange): Call notifyMutedObservers on the
next runloop so events aren't dispatched synchronously.

* testing/Internals.cpp:
(WebCore::Internals::setPageMuted): Add new state.

Source/WebKit:

* UIProcess/API/C/WKPagePrivate.h: Add kWKMediaScreenCaptureMuted.

* UIProcess/API/Cocoa/WKWebView.mm:
(-[WKWebView _setPageMuted:]):
(-[WKWebView _setMediaCaptureMuted:]): Deleted, it was unused.
(-[WKWebView _muteMediaCapture]): Deleted, it was unused.
* UIProcess/API/Cocoa/WKWebViewPrivate.h:

* UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
(WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionValidRequest): Fix
a log comment typo.

* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::setMediaStreamCaptureMuted): Use the new flag.
(WebKit::WebPageProxy::activateMediaStreamCaptureInPage): Ditto.
(WebKit::WebPageProxy::setMuted): Ditto.
* UIProcess/WebPageProxy.h:
(WebKit::WebPageProxy::isMediaStreamCaptureMuted const): Ditto.

LayoutTests:

* fast/mediastream/media-stream-page-muted-expected.txt: Added.
* fast/mediastream/media-stream-page-muted.html: Added.
* fast/mediastream/media-stream-track-interrupted.html: Read page muted state before
interrupting capture track because the page state now changes immediately.
* fast/mediastream/media-stream-track-muted.html: Ditto, for muting.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@243899 268f45cc-cd09-0410-ab3c-d52691b4dbfc

22 files changed:
LayoutTests/ChangeLog
LayoutTests/fast/mediastream/media-stream-page-muted-expected.txt [new file with mode: 0644]
LayoutTests/fast/mediastream/media-stream-page-muted.html [new file with mode: 0644]
LayoutTests/fast/mediastream/media-stream-track-interrupted.html
LayoutTests/fast/mediastream/media-stream-track-muted.html
Source/WebCore/ChangeLog
Source/WebCore/Modules/mediastream/MediaStream.cpp
Source/WebCore/Modules/mediastream/MediaStreamTrack.cpp
Source/WebCore/Modules/mediastream/MediaStreamTrack.h
Source/WebCore/page/MediaProducer.h
Source/WebCore/page/Page.h
Source/WebCore/platform/mediastream/MediaStreamPrivate.cpp
Source/WebCore/platform/mediastream/MediaStreamPrivate.h
Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
Source/WebCore/testing/Internals.cpp
Source/WebKit/ChangeLog
Source/WebKit/UIProcess/API/C/WKPagePrivate.h
Source/WebKit/UIProcess/API/Cocoa/WKWebView.mm
Source/WebKit/UIProcess/API/Cocoa/WKWebViewPrivate.h
Source/WebKit/UIProcess/UserMediaPermissionRequestManagerProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.h

index 55c4601..82c27f5 100644 (file)
@@ -1,3 +1,17 @@
+2019-04-04  Eric Carlson  <eric.carlson@apple.com>
+
+        [MediaStream] Host should be able to mute screen capture and camera/microphone independently
+        https://bugs.webkit.org/show_bug.cgi?id=196555
+        <rdar://problem/47303865>
+
+        Reviewed by Youenn Fablet.
+
+        * fast/mediastream/media-stream-page-muted-expected.txt: Added.
+        * fast/mediastream/media-stream-page-muted.html: Added.
+        * fast/mediastream/media-stream-track-interrupted.html: Read page muted state before
+        interrupting capture track because the page state now changes immediately.
+        * fast/mediastream/media-stream-track-muted.html: Ditto, for muting.
+
 2019-04-04  Antti Koivisto  <antti@apple.com>
 
         Compute accurate regions for touch-action
diff --git a/LayoutTests/fast/mediastream/media-stream-page-muted-expected.txt b/LayoutTests/fast/mediastream/media-stream-page-muted-expected.txt
new file mode 100644 (file)
index 0000000..93e7dec
--- /dev/null
@@ -0,0 +1,66 @@
+
+*** Creating screen capture stream
+PASS displayStream is an instance of Object
+PASS displayStream.getTracks().length is 1
+PASS displayStream.getVideoTracks().length is 1
+PASS screenCaptureTrack.muted is false
+
+*** Creating camera and microphone stream
+PASS cameraStream is an instance of Object
+PASS cameraStream.getTracks().length is 2
+PASS microphoneCaptureTrack.muted is false
+PASS cameraCaptureTrack.muted is false
+
+*** Muting screen capture
+EVENT: mute
+PASS screenCaptureTrack.muted is true
+PASS microphoneCaptureTrack.muted is false
+PASS cameraCaptureTrack.muted is false
+PASS window.internals.pageMediaState().includes(HasMutedDisplayCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasActiveDisplayCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasActiveAudioCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasMutedAudioCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasActiveAudioCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasMutedAudioCaptureDevice) became false
+
+*** Muting camera and microphone
+EVENT: mute
+EVENT: mute
+PASS screenCaptureTrack.muted is true
+PASS microphoneCaptureTrack.muted is true
+PASS cameraCaptureTrack.muted is true
+PASS window.internals.pageMediaState().includes(HasMutedDisplayCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasActiveDisplayCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasMutedAudioCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasActiveAudioCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasMutedVideoCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasActiveVideoCaptureDevice) became false
+
+*** Unmuting camera and microphone
+EVENT: unmute
+EVENT: unmute
+PASS screenCaptureTrack.muted is true
+PASS microphoneCaptureTrack.muted is false
+PASS cameraCaptureTrack.muted is false
+PASS window.internals.pageMediaState().includes(HasMutedDisplayCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasActiveDisplayCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasActiveAudioCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasMutedAudioCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasActiveAudioCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasMutedAudioCaptureDevice) became false
+
+*** Unmuting screen capture
+EVENT: unmute
+PASS screenCaptureTrack.muted is false
+PASS microphoneCaptureTrack.muted is false
+PASS cameraCaptureTrack.muted is false
+PASS window.internals.pageMediaState().includes(HasActiveDisplayCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasMutedDisplayCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasActiveAudioCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasMutedAudioCaptureDevice) became false
+PASS window.internals.pageMediaState().includes(HasActiveAudioCaptureDevice) became true
+PASS window.internals.pageMediaState().includes(HasMutedAudioCaptureDevice) became false
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
diff --git a/LayoutTests/fast/mediastream/media-stream-page-muted.html b/LayoutTests/fast/mediastream/media-stream-page-muted.html
new file mode 100644 (file)
index 0000000..b277ca6
--- /dev/null
@@ -0,0 +1,195 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <title>mediastream page muted</title>
+    <script src="../../resources/js-test-pre.js"></script>
+    <script>
+        async function checkPageState(activeState, inactiveState) {
+            await new Promise((resolve, reject) => {
+                let retryCount = 0;
+                let retryInterval = 100;
+                let maximumRetryCount = 20 * 1000 / retryInterval;
+
+                let test = () => {
+
+                    if ( window.internals
+                        && window.internals.pageMediaState().includes(activeState)
+                        && !window.internals.pageMediaState().includes(inactiveState)) {
+
+                        testPassed(`window.internals.pageMediaState().includes(${activeState}) became true`);
+                        testPassed(`window.internals.pageMediaState().includes(${inactiveState}) became false`);
+
+                        resolve()
+                        return;
+                    }
+
+                    if (++retryCount > maximumRetryCount) {
+                        testFailed(`Page muted state failed to change after ${maximumRetryCount / 1000 * retryInterval} seconds`);
+                        resolve();
+                        return;
+                    }
+
+                    setTimeout(test, retryInterval);
+                }
+
+                setTimeout(test, 0);
+            });
+        }
+
+        async function waitForMutedChange(stream, event, mute, unmute) {
+            await new Promise((resolve, reject) => {
+                const interval = 200;
+                let timeout = setTimeout(() => {
+                    testFailed(`'${event}' event not fired in ${interval / 1000} seconds!`);
+                    resolve();
+                }, interval);
+
+                let checkResult = (callback) => {
+                    switch (callback()) {
+                    case "resolve":
+                        clearTimeout(timeout);
+                        resolve();
+                        break;
+                    case "reject":
+                        clearTimeout(timeout);
+                        reject();
+                        break;
+                    case "continue":
+                        break;
+                    }
+                }
+
+                stream.getTracks().forEach(track => {
+                    track.onmute = (evt) => { debug('EVENT: mute'); checkResult(mute); }
+                    track.onunmute = (evt) => { debug('EVENT: unmute'); checkResult(unmute); }
+                });
+            });
+        }
+
+        async function createScreenCaptureStream() {
+            debug("<br>*** Creating screen capture stream");
+                       displayStream = await navigator.mediaDevices.getDisplayMedia({ video: true });
+            shouldBeType("displayStream", "Object");
+            shouldBe("displayStream.getTracks().length", "1");
+            shouldBe("displayStream.getVideoTracks().length", "1");
+            screenCaptureTrack = displayStream.getVideoTracks()[0];
+            shouldBeFalse("screenCaptureTrack.muted");
+        }
+
+        async function createCameraMicrophoneStream() {
+            debug("<br>*** Creating camera and microphone stream");
+            cameraStream = await navigator.mediaDevices.getUserMedia({ audio: true, video: true });
+            shouldBeType("cameraStream", "Object");
+            shouldBe("cameraStream.getTracks().length", "2");
+            microphoneCaptureTrack = cameraStream.getAudioTracks()[0];
+            cameraCaptureTrack = cameraStream.getVideoTracks()[0];
+            shouldBeFalse("microphoneCaptureTrack.muted");
+            shouldBeFalse("cameraCaptureTrack.muted");
+        }
+
+        async function muteScreenCapture() {
+            debug("<br>*** Muting screen capture");
+            if (window.internals)
+                window.internals.setPageMuted("screencapture");
+
+            await waitForMutedChange(displayStream, "mute",
+                () => { return "resolve"; },
+                () => { testFailed("unexpected 'unmute' event"); return "reject"; }
+            );
+
+            shouldBeTrue("screenCaptureTrack.muted");
+            shouldBeFalse("microphoneCaptureTrack.muted");
+            shouldBeFalse("cameraCaptureTrack.muted");
+
+            await checkPageState("HasMutedDisplayCaptureDevice", "HasActiveDisplayCaptureDevice");
+            await checkPageState("HasActiveAudioCaptureDevice", "HasMutedAudioCaptureDevice");
+            await checkPageState("HasActiveAudioCaptureDevice", "HasMutedAudioCaptureDevice");
+        }
+
+        async function muteCameraMicrophone() {
+            debug("<br>*** Muting camera and microphone");
+            if (window.internals)
+                window.internals.setPageMuted("screencapture,capturedevices");
+
+            count = 0;
+            await waitForMutedChange(cameraStream, "mute",
+                () => { return (++count == 2) ? "resolve" : "continue"; },
+                () => { testFailed("unexpected 'unmute' event"); return "reject"; }
+            );
+            shouldBeTrue("screenCaptureTrack.muted");
+            shouldBeTrue("microphoneCaptureTrack.muted");
+            shouldBeTrue("cameraCaptureTrack.muted");
+
+            await checkPageState("HasMutedDisplayCaptureDevice", "HasActiveDisplayCaptureDevice");
+            await checkPageState("HasMutedAudioCaptureDevice", "HasActiveAudioCaptureDevice");
+            await checkPageState("HasMutedVideoCaptureDevice", "HasActiveVideoCaptureDevice");
+        }
+
+        async function unmuteCameraMicrophone() {
+            debug("<br>*** Unmuting camera and microphone");
+            if (window.internals)
+                internals.setPageMuted("screencapture");
+
+            count = 0;
+            await waitForMutedChange(cameraStream, "unmute",
+                () => { testFailed("unexpected 'mute' event"); return "reject"; },
+                () => { return (++count == 2) ? "resolve" : "continue"; }
+            );
+            shouldBeTrue("screenCaptureTrack.muted");
+            shouldBeFalse("microphoneCaptureTrack.muted");
+            shouldBeFalse("cameraCaptureTrack.muted");
+
+            await checkPageState("HasMutedDisplayCaptureDevice", "HasActiveDisplayCaptureDevice");
+            await checkPageState("HasActiveAudioCaptureDevice", "HasMutedAudioCaptureDevice");
+            await checkPageState("HasActiveAudioCaptureDevice", "HasMutedAudioCaptureDevice");
+        }
+
+        async function unmuteScreenCapture() {
+            debug("<br>*** Unmuting screen capture");
+            if (window.internals)
+                internals.setPageMuted("");
+
+            await waitForMutedChange(displayStream, "unmute",
+                () => { testFailed("unexpected 'mute' event"); return "reject"; },
+                () => { return "resolve"; }
+            );
+            shouldBeFalse("screenCaptureTrack.muted");
+            shouldBeFalse("microphoneCaptureTrack.muted");
+            shouldBeFalse("cameraCaptureTrack.muted");
+
+            await checkPageState("HasActiveDisplayCaptureDevice", "HasMutedDisplayCaptureDevice");
+            await checkPageState("HasActiveAudioCaptureDevice", "HasMutedAudioCaptureDevice");
+            await checkPageState("HasActiveAudioCaptureDevice", "HasMutedAudioCaptureDevice");
+        }
+
+        jsTestIsAsync = true;
+        if (window.testRunner) {
+            testRunner.setUserMediaPermission(true);
+            testRunner.waitUntilDone();
+        }
+        if (window.internals) {
+            window.internals.setMockMediaCaptureDevicesEnabled(true);
+            window.internals.settings.setScreenCaptureEnabled(true);
+        }
+
+        (async function() {
+            try {
+                await createScreenCaptureStream();
+                await createCameraMicrophoneStream();
+                await muteScreenCapture();
+                await muteCameraMicrophone();
+                await unmuteCameraMicrophone();
+                await unmuteScreenCapture();
+            } catch (exception) {
+                failTest(exception.description);
+            }
+
+            finishJSTest();
+        })()
+
+    </script>
+</head>
+<body>
+<script src="../../resources/js-test-post.js"></script>
+</body>
+</html>
index 7b0b41d..833f6aa 100644 (file)
                 track.onunmute = () => reject("Got 'unmute' event unexpectedly!");
                 track.onmute = () => {
                     new Promise((innerResolve, innerReject) => {
-                        waitForPageStateChange(10, internals.pageMediaState(), innerResolve, innerReject)
+                        waitForPageStateChange(10, pageState, innerResolve, innerReject)
                     }).then((pageMediaState) => {
 
                         track.onunmute = (evt) => {
-                            waitForPageStateChange(10, internals.pageMediaState(), resolve, reject)
+                            waitForPageStateChange(10, pageState, resolve, reject)
                         }
 
                         if (window.internals) {
                             assert_false(pageMediaState.includes(isVideo ? 'HasMutedAudioCaptureDevice' : 'HasMutedVideoCaptureDevice'));
                             assert_true(pageMediaState.includes(isVideo ? 'HasActiveAudioCaptureDevice' : 'HasActiveVideoCaptureDevice'));
                             assert_false(pageMediaState.includes(isVideo ? 'HasActiveVideoCaptureDevice' : 'HasActiveAudioCaptureDevice'));
+                            pageState = internals.pageMediaState();
                             internals.setMediaStreamSourceInterrupted(track, false)
                         }
                     })
                 }
 
-                if (window.internals)
+                if (window.internals) {
+                    pageState = internals.pageMediaState();
                     internals.setMediaStreamSourceInterrupted(track, true);
+                }
                 setTimeout(() => reject("Muted state did not change in 1 second"), 1000);
             });
         }, title);
index 81ee395..24d3dbf 100644 (file)
                 track.onunmute = () => reject("Got 'unmute' event unexpectedly!");
                 track.onmute = () => {
                     new Promise((innerResolve, innerReject) => {
-                        waitForPageStateChange(10, internals.pageMediaState(), innerResolve, innerReject)
+                        waitForPageStateChange(10, pageState, innerResolve, innerReject)
                     }).then((pageMediaState) => {
 
                         track.onunmute = (evt) => {
-                            waitForPageStateChange(10, internals.pageMediaState(), resolve, reject)
+                            waitForPageStateChange(10, pageState, resolve, reject)
                         }
 
                         if (window.internals) {
                             assert_false(pageMediaState.includes(isVideo ? 'HasMutedAudioCaptureDevice' : 'HasMutedVideoCaptureDevice'));
                             assert_true(pageMediaState.includes(isVideo ? 'HasActiveAudioCaptureDevice' : 'HasActiveVideoCaptureDevice'));
                             assert_false(pageMediaState.includes(isVideo ? 'HasActiveVideoCaptureDevice' : 'HasActiveAudioCaptureDevice'));
+                            pageState = internals.pageMediaState();
                             internals.setMediaStreamTrackMuted(track, false)
                         }
                     })
                 }
 
-                if (window.internals)
+                if (window.internals) {
+                    pageState = internals.pageMediaState();
                     internals.setMediaStreamTrackMuted(track, true);
+                }
                 setTimeout(() => reject("Muted state did not change in .5 second"), 500);
             });
         }, title);
@@ -71,7 +74,6 @@
             });
     }, "Create stream");
 
-
     </script>
 </head>
 <body>
index 889d12f..6819ac2 100644 (file)
@@ -1,3 +1,45 @@
+2019-04-04  Eric Carlson  <eric.carlson@apple.com>
+
+        [MediaStream] Host should be able to mute screen capture and camera/microphone independently
+        https://bugs.webkit.org/show_bug.cgi?id=196555
+        <rdar://problem/47303865>
+
+        Reviewed by Youenn Fablet.
+
+        Test: fast/mediastream/media-stream-page-muted.html
+
+        * Modules/mediastream/MediaStream.cpp:
+        (WebCore::MediaStream::MediaStream): Call setCaptureTracksMuted to pass page muted 
+        state to tracks.
+        (WebCore::MediaStream::startProducingData): Ditto.
+        (WebCore::MediaStream::setCaptureTracksMuted): New.
+        * Modules/mediastream/MediaStream.h:
+
+        * Modules/mediastream/MediaStreamTrack.cpp:
+        (WebCore::MediaStreamTrack::MediaStreamTrack): Call setMuted with page muted state.
+        (WebCore::MediaStreamTrack::setMuted): Set muted according to page state and source type.
+        (WebCore::MediaStreamTrack::pageMutedStateDidChange): Call setMuted.
+        (WebCore::MediaStreamTrack::mediaState const): Update for new page state.
+        * Modules/mediastream/MediaStreamTrack.h:
+
+        * page/MediaProducer.h: Split capture muted state into two: camera/microphone and screen.
+
+        * page/Page.h:
+        (WebCore::Page::isMediaCaptureMuted const): Update for state changes.
+
+        * platform/mediastream/MediaStreamPrivate.cpp:
+        (WebCore::MediaStreamPrivate::setCaptureTracksMuted): Deleted.
+        (WebCore::MediaStreamPrivate::hasCaptureVideoSource const): Deleted.
+        * platform/mediastream/MediaStreamPrivate.h:
+
+        * platform/mediastream/RealtimeMediaSource.cpp:
+        (WebCore::RealtimeMediaSource::setMuted): Log state.
+        (WebCore::RealtimeMediaSource::notifyMutedChange): Call notifyMutedObservers on the
+        next runloop so events aren't dispatched synchronously.
+
+        * testing/Internals.cpp:
+        (WebCore::Internals::setPageMuted): Add new state.
+
 2019-04-04  Chris Dumez  <cdumez@apple.com>
 
         Unreviewed, update r243884 to use macros in Compiler.h instead.
index 51e3402..6f3965d 100644 (file)
@@ -119,8 +119,6 @@ MediaStream::MediaStream(ScriptExecutionContext& context, Ref<MediaStreamPrivate
     m_private->setLogger(logger(), logIdentifier());
 #endif
     setIsActive(m_private->active());
-    if (document()->page() && document()->page()->isMediaCaptureMuted())
-        m_private->setCaptureTracksMuted(true);
     m_private->addObserver(*this);
     MediaStreamRegistry::shared().registerStream(*this);
 
@@ -326,9 +324,6 @@ void MediaStream::startProducingData()
 
     m_mediaSession->canProduceAudioChanged();
     m_private->startProducingData();
-
-    if (document->page()->isMediaCaptureMuted())
-        m_private->setCaptureTracksMuted(true);
 }
 
 void MediaStream::stopProducingData()
index 72acc40..17284b9 100644 (file)
@@ -71,8 +71,11 @@ MediaStreamTrack::MediaStreamTrack(ScriptExecutionContext& context, Ref<MediaStr
 #endif
     m_private->addObserver(*this);
 
-    if (auto document = this->document())
+    if (auto document = this->document()) {
         document->addAudioProducer(*this);
+        if (isCaptureTrack() && document->page() && document->page()->mutedState())
+            setMuted(document->page()->mutedState());
+    }
 }
 
 MediaStreamTrack::~MediaStreamTrack()
@@ -171,6 +174,26 @@ bool MediaStreamTrack::muted() const
     return m_private->muted();
 }
 
+void MediaStreamTrack::setMuted(MediaProducer::MutedStateFlags state)
+{
+    bool trackMuted = false;
+    switch (source().deviceType()) {
+    case CaptureDevice::DeviceType::Microphone:
+    case CaptureDevice::DeviceType::Camera:
+        trackMuted = state & AudioAndVideoCaptureIsMuted;
+        break;
+    case CaptureDevice::DeviceType::Screen:
+    case CaptureDevice::DeviceType::Window:
+        trackMuted = state & ScreenCaptureIsMuted;
+        break;
+    case CaptureDevice::DeviceType::Unknown:
+        ASSERT_NOT_REACHED();
+        break;
+    }
+
+    m_private->setMuted(trackMuted);
+}
+
 auto MediaStreamTrack::readyState() const -> State
 {
     return ended() ? State::Ended : State::Live;
@@ -373,7 +396,7 @@ void MediaStreamTrack::pageMutedStateDidChange()
     if (!document || !document->page())
         return;
 
-    m_private->setMuted(document->page()->isMediaCaptureMuted());
+    setMuted(document->page()->mutedState());
 }
 
 MediaProducer::MediaStateFlags MediaStreamTrack::mediaState() const
@@ -385,10 +408,8 @@ MediaProducer::MediaStateFlags MediaStreamTrack::mediaState() const
     if (!document || !document->page())
         return IsNotPlaying;
 
-    bool pageCaptureMuted = document->page()->isMediaCaptureMuted();
-
     if (source().type() == RealtimeMediaSource::Type::Audio) {
-        if (source().interrupted() && !pageCaptureMuted)
+        if (source().interrupted() && !source().muted())
             return HasInterruptedAudioCaptureDevice;
         if (muted())
             return HasMutedAudioCaptureDevice;
@@ -397,7 +418,7 @@ MediaProducer::MediaStateFlags MediaStreamTrack::mediaState() const
     } else {
         auto deviceType = source().deviceType();
         ASSERT(deviceType == CaptureDevice::DeviceType::Camera || deviceType == CaptureDevice::DeviceType::Screen || deviceType == CaptureDevice::DeviceType::Window);
-        if (source().interrupted() && !pageCaptureMuted)
+        if (source().interrupted() && !source().muted())
             return deviceType == CaptureDevice::DeviceType::Camera ? HasInterruptedVideoCaptureDevice : HasInterruptedDisplayCaptureDevice;
         if (muted())
             return deviceType == CaptureDevice::DeviceType::Camera ? HasMutedVideoCaptureDevice : HasMutedDisplayCaptureDevice;
@@ -442,8 +463,10 @@ void MediaStreamTrack::trackMutedChanged(MediaStreamTrackPrivate&)
     if (scriptExecutionContext()->activeDOMObjectsAreSuspended() || scriptExecutionContext()->activeDOMObjectsAreStopped() || m_ended)
         return;
 
-    AtomicString eventType = muted() ? eventNames().muteEvent : eventNames().unmuteEvent;
-    dispatchEvent(Event::create(eventType, Event::CanBubble::No, Event::IsCancelable::No));
+    m_eventTaskQueue.enqueueTask([this, muted = this->muted()] {
+        AtomicString eventType = muted ? eventNames().muteEvent : eventNames().unmuteEvent;
+        dispatchEvent(Event::create(eventType, Event::CanBubble::No, Event::IsCancelable::No));
+    });
 
     configureTrackRendering();
 }
index e3d1cb6..7c11f3f 100644 (file)
@@ -82,6 +82,7 @@ public:
     void setEnabled(bool);
 
     bool muted() const;
+    void setMuted(MediaProducer::MutedStateFlags);
 
     enum class State { Live, Ended };
     State readyState() const;
@@ -199,6 +200,7 @@ private:
     MediaTrackConstraints m_constraints;
     Optional<DOMPromiseDeferred<void>> m_promise;
     GenericTaskQueue<ScriptExecutionContext> m_taskQueue;
+    GenericTaskQueue<Timer> m_eventTaskQueue;
 
     bool m_ended { false };
 };
index d6488af..b01ab94 100644 (file)
@@ -56,6 +56,7 @@ public:
         AudioCaptureMask = HasActiveAudioCaptureDevice | HasMutedAudioCaptureDevice | HasInterruptedAudioCaptureDevice,
         VideoCaptureMask = HasActiveVideoCaptureDevice | HasMutedVideoCaptureDevice | HasInterruptedVideoCaptureDevice,
         DisplayCaptureMask = HasActiveDisplayCaptureDevice | HasMutedDisplayCaptureDevice | HasInterruptedDisplayCaptureDevice,
+        MutedCaptureMask =  HasMutedAudioCaptureDevice | HasMutedVideoCaptureDevice | HasMutedDisplayCaptureDevice,
         MediaCaptureMask = AudioCaptureMask | VideoCaptureMask | DisplayCaptureMask,
     };
     typedef unsigned MediaStateFlags;
@@ -67,7 +68,10 @@ public:
     enum MutedState {
         NoneMuted = 0,
         AudioIsMuted = 1 << 0,
-        CaptureDevicesAreMuted = 1 << 1,
+        AudioAndVideoCaptureIsMuted = 1 << 1,
+        ScreenCaptureIsMuted = 1 << 2,
+
+        MediaStreamCaptureIsMuted = AudioAndVideoCaptureIsMuted | ScreenCaptureIsMuted,
     };
     typedef unsigned MutedStateFlags;
 
index 3e4e516..7ef211d 100644 (file)
@@ -621,7 +621,7 @@ public:
     void updateIsPlayingMedia(uint64_t);
     MediaProducer::MutedStateFlags mutedState() const { return m_mutedState; }
     bool isAudioMuted() const { return m_mutedState & MediaProducer::AudioIsMuted; }
-    bool isMediaCaptureMuted() const { return m_mutedState & MediaProducer::CaptureDevicesAreMuted; };
+    bool isMediaCaptureMuted() const { return m_mutedState & MediaProducer::MediaStreamCaptureIsMuted; };
     void schedulePlaybackControlsManagerUpdate();
     WEBCORE_EXPORT void setMuted(MediaProducer::MutedStateFlags);
     WEBCORE_EXPORT void stopMediaCapture();
index 4a73d9e..c956f11 100644 (file)
@@ -193,15 +193,6 @@ bool MediaStreamPrivate::isProducingData() const
     return false;
 }
 
-void MediaStreamPrivate::setCaptureTracksMuted(bool muted)
-{
-    ALWAYS_LOG(LOGIDENTIFIER, muted);
-    for (auto& track : m_trackSet.values()) {
-        if (track->isCaptureTrack())
-            track->setMuted(muted);
-    }
-}
-
 bool MediaStreamPrivate::hasVideo() const
 {
     for (auto& track : m_trackSet.values()) {
index e2da353..76b4679 100644 (file)
@@ -105,7 +105,6 @@ public:
 
     bool hasCaptureVideoSource() const;
     bool hasCaptureAudioSource() const;
-    void setCaptureTracksMuted(bool);
 
     FloatSize intrinsicSize() const;
 
index a9d2946..e9757e2 100644 (file)
@@ -91,7 +91,7 @@ void RealtimeMediaSource::setInterrupted(bool interrupted, bool pageMuted)
 
 void RealtimeMediaSource::setMuted(bool muted)
 {
-    ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER);
+    ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, muted);
 
     if (muted)
         stop();
@@ -110,7 +110,7 @@ void RealtimeMediaSource::notifyMutedChange(bool muted)
     if (m_muted == muted)
         return;
 
-    ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER);
+    ALWAYS_LOG_IF(m_logger, LOGIDENTIFIER, muted);
     m_muted = muted;
 
     notifyMutedObservers();
index c0f2e50..4f4d119 100644 (file)
@@ -4052,7 +4052,9 @@ void Internals::setPageMuted(StringView statesString)
         if (equalLettersIgnoringASCIICase(stateString, "audio"))
             state |= MediaProducer::AudioIsMuted;
         if (equalLettersIgnoringASCIICase(stateString, "capturedevices"))
-            state |= MediaProducer::CaptureDevicesAreMuted;
+            state |= MediaProducer::AudioAndVideoCaptureIsMuted;
+        if (equalLettersIgnoringASCIICase(stateString, "screencapture"))
+            state |= MediaProducer::ScreenCaptureIsMuted;
     }
 
     if (Page* page = document->page())
index eb85379..5c6549d 100644 (file)
@@ -1,3 +1,30 @@
+2019-04-04  Eric Carlson  <eric.carlson@apple.com>
+
+        [MediaStream] Host should be able to mute screen capture and camera/microphone independently
+        https://bugs.webkit.org/show_bug.cgi?id=196555
+        <rdar://problem/47303865>
+
+        Reviewed by Youenn Fablet.
+
+        * UIProcess/API/C/WKPagePrivate.h: Add kWKMediaScreenCaptureMuted.
+
+        * UIProcess/API/Cocoa/WKWebView.mm:
+        (-[WKWebView _setPageMuted:]):
+        (-[WKWebView _setMediaCaptureMuted:]): Deleted, it was unused.
+        (-[WKWebView _muteMediaCapture]): Deleted, it was unused.
+        * UIProcess/API/Cocoa/WKWebViewPrivate.h:
+
+        * UIProcess/UserMediaPermissionRequestManagerProxy.cpp:
+        (WebKit::UserMediaPermissionRequestManagerProxy::processUserMediaPermissionValidRequest): Fix
+        a log comment typo.
+
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::setMediaStreamCaptureMuted): Use the new flag.
+        (WebKit::WebPageProxy::activateMediaStreamCaptureInPage): Ditto.
+        (WebKit::WebPageProxy::setMuted): Ditto.
+        * UIProcess/WebPageProxy.h:
+        (WebKit::WebPageProxy::isMediaStreamCaptureMuted const): Ditto.
+
 2019-04-04  Per Arne Vollan  <pvollan@apple.com>
 
         [macOS][iOS] Add filter to syscall sandbox rule
index abdd361..b076e75 100644 (file)
@@ -126,6 +126,7 @@ enum {
     kWKMediaNoneMuted = 0,
     kWKMediaAudioMuted = 1 << 0,
     kWKMediaCaptureDevicesMuted = 1 << 1,
+    kWKMediaScreenCaptureMuted = 1 << 2,
 };
 typedef uint32_t WKMediaMutedState;
 WK_EXPORT void WKPageSetMuted(WKPageRef page, WKMediaMutedState muted);
index cf40032..afb85cd 100644 (file)
@@ -5776,16 +5776,6 @@ static inline WebKit::FindOptions toFindOptions(_WKFindOptions wkFindOptions)
     return WebKit::toWKMediaCaptureState(_page->mediaStateFlags());
 }
 
-- (void)_setMediaCaptureMuted:(BOOL)muted
-{
-    _page->setMediaStreamCaptureMuted(muted);
-}
-
-- (void)_muteMediaCapture
-{
-    _page->setMediaStreamCaptureMuted(true);
-}
-
 - (void)_setMediaCaptureEnabled:(BOOL)enabled
 {
     _page->setMediaCaptureEnabled(enabled);
@@ -5803,7 +5793,9 @@ static inline WebKit::FindOptions toFindOptions(_WKFindOptions wkFindOptions)
     if (mutedState & _WKMediaAudioMuted)
         coreState |= WebCore::MediaProducer::AudioIsMuted;
     if (mutedState & _WKMediaCaptureDevicesMuted)
-        coreState |= WebCore::MediaProducer::CaptureDevicesAreMuted;
+        coreState |= WebCore::MediaProducer::AudioAndVideoCaptureIsMuted;
+    if (mutedState & _WKMediaScreenCaptureMuted)
+        coreState |= WebCore::MediaProducer::ScreenCaptureIsMuted;
 
     _page->setMuted(coreState);
 }
index a86d145..bd344c2 100644 (file)
@@ -53,6 +53,7 @@ typedef NS_OPTIONS(NSUInteger, _WKMediaMutedState) {
     _WKMediaNoneMuted = 0,
     _WKMediaAudioMuted = 1 << 0,
     _WKMediaCaptureDevicesMuted = 1 << 1,
+    _WKMediaScreenCaptureMuted = 1 << 2,
 } WK_API_AVAILABLE(macos(10.13), ios(11.0));
 
 typedef NS_OPTIONS(NSUInteger, _WKCaptureDevices) {
@@ -407,8 +408,6 @@ typedef NS_OPTIONS(NSUInteger, _WKRectEdge) {
 
 @property (nonatomic, readonly) _WKMediaCaptureState _mediaCaptureState WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
 
-- (void)_setMediaCaptureMuted:(BOOL)muted WK_API_AVAILABLE(macos(WK_MAC_TBA), ios(WK_IOS_TBA));
-- (void)_muteMediaCapture WK_API_AVAILABLE(macos(10.13), ios(11.0));
 - (void)_setPageMuted:(_WKMediaMutedState)mutedState WK_API_AVAILABLE(macos(10.13), ios(11.0));
 
 @property (nonatomic, setter=_setMediaCaptureEnabled:) BOOL _mediaCaptureEnabled WK_API_AVAILABLE(macos(10.13), ios(11.0));
index 1993372..03a9a14 100644 (file)
@@ -429,7 +429,7 @@ void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionInvalidRe
 
 void UserMediaPermissionRequestManagerProxy::processUserMediaPermissionValidRequest(Ref<UserMediaPermissionRequestProxy>&& request, Vector<CaptureDevice>&& audioDevices, Vector<CaptureDevice>&& videoDevices, String&& deviceIdentifierHashSalt)
 {
-    ALWAYS_LOG(LOGIDENTIFIER, request->userMediaID(), ", video:, ", videoDevices.size(), " audio: ", audioDevices.size());
+    ALWAYS_LOG(LOGIDENTIFIER, request->userMediaID(), ", video: ", videoDevices.size(), " audio: ", audioDevices.size());
     if (videoDevices.isEmpty() && audioDevices.isEmpty()) {
         denyRequest(request->userMediaID(), UserMediaPermissionRequestProxy::UserMediaAccessDenialReason::NoConstraints, emptyString());
         return;
index 67c76fc..42f43f2 100644 (file)
@@ -2056,9 +2056,9 @@ void WebPageProxy::setEditable(bool editable)
 void WebPageProxy::setMediaStreamCaptureMuted(bool muted)
 {
     if (muted)
-        setMuted(m_mutedState | WebCore::MediaProducer::CaptureDevicesAreMuted);
+        setMuted(m_mutedState | WebCore::MediaProducer::MediaStreamCaptureIsMuted);
     else
-        setMuted(m_mutedState & ~WebCore::MediaProducer::CaptureDevicesAreMuted);
+        setMuted(m_mutedState & ~WebCore::MediaProducer::MediaStreamCaptureIsMuted);
 }
 
 void WebPageProxy::activateMediaStreamCaptureInPage()
@@ -2066,7 +2066,7 @@ void WebPageProxy::activateMediaStreamCaptureInPage()
 #if ENABLE(MEDIA_STREAM)
     UserMediaProcessManager::singleton().muteCaptureMediaStreamsExceptIn(*this);
 #endif
-    setMuted(m_mutedState & ~WebCore::MediaProducer::CaptureDevicesAreMuted);
+    setMuted(m_mutedState & ~WebCore::MediaProducer::MediaStreamCaptureIsMuted);
 }
 
 #if !PLATFORM(IOS_FAMILY)
@@ -5348,8 +5348,8 @@ void WebPageProxy::setMuted(WebCore::MediaProducer::MutedStateFlags state)
         return;
 
 #if ENABLE(MEDIA_STREAM)
-    bool hasMutedCaptureStreams = m_mediaState & (WebCore::MediaProducer::HasMutedAudioCaptureDevice | WebCore::MediaProducer::HasMutedVideoCaptureDevice);
-    if (hasMutedCaptureStreams && !(state & WebCore::MediaProducer::CaptureDevicesAreMuted))
+    bool hasMutedCaptureStreams = m_mediaState & WebCore::MediaProducer::MutedCaptureMask;
+    if (hasMutedCaptureStreams && !(state & WebCore::MediaProducer::MediaStreamCaptureIsMuted))
         UserMediaProcessManager::singleton().muteCaptureMediaStreamsExceptIn(*this);
 #endif
 
index 60185a9..0b22019 100644 (file)
@@ -614,7 +614,7 @@ public:
     bool isEditable() const { return m_isEditable; }
 
     void activateMediaStreamCaptureInPage();
-    bool isMediaStreamCaptureMuted() const { return m_mutedState & WebCore::MediaProducer::CaptureDevicesAreMuted; }
+    bool isMediaStreamCaptureMuted() const { return m_mutedState & WebCore::MediaProducer::MediaStreamCaptureIsMuted; }
     void setMediaStreamCaptureMuted(bool);
     void executeEditCommand(const String& commandName, const String& argument, WTF::Function<void(CallbackBase::Error)>&&);