Media controls should not be displayed for a video until it starts playing
authorwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 25 Jul 2016 20:51:43 +0000 (20:51 +0000)
committerwenson_hsieh@apple.com <wenson_hsieh@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 25 Jul 2016 20:51:43 +0000 (20:51 +0000)
https://bugs.webkit.org/show_bug.cgi?id=160092
<rdar://problem/26986673>

Reviewed by Beth Dakin.

Source/WebCore:

For videos that have never played back yet, we should not show media controls. To ensure this
behavior, we ensure that the playback behavior restriction is set upon creating the media
element. This restriction is then removed when the media element begins to play.

Added two new WebKit API tests.

* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::HTMLMediaElement):

Tools:

Verify that multiple videos do or don't show the media controller depending on whether videos
are playing. Also tweaks an existing API test (VideoControlsManagerSingleLargeVideo) that was
passing because we were always showing media controls for large videos with audio, even if they
had not played back yet. This change ensures that large videos with audio show media controls
only after they begin to play back, and not by virtue of being large enough for main content.

* TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
* TestWebKitAPI/Tests/WebKit2Cocoa/VideoControlsManager.mm:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WebKit2Cocoa/large-video-with-audio.html:
* TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio-autoplay.html: Added.
* TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@203698 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Source/WebCore/ChangeLog
Source/WebCore/html/HTMLMediaElement.cpp
Tools/ChangeLog
Tools/TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/VideoControlsManager.mm
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/large-video-with-audio.html
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio-autoplay.html [new file with mode: 0644]
Tools/TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio.html [new file with mode: 0644]

index e2bc833..66b16e5 100644 (file)
@@ -1,3 +1,20 @@
+2016-07-25  Wenson Hsieh  <wenson_hsieh@apple.com>
+
+        Media controls should not be displayed for a video until it starts playing
+        https://bugs.webkit.org/show_bug.cgi?id=160092
+        <rdar://problem/26986673>
+
+        Reviewed by Beth Dakin.
+
+        For videos that have never played back yet, we should not show media controls. To ensure this
+        behavior, we ensure that the playback behavior restriction is set upon creating the media
+        element. This restriction is then removed when the media element begins to play.
+
+        Added two new WebKit API tests.
+
+        * html/HTMLMediaElement.cpp:
+        (WebCore::HTMLMediaElement::HTMLMediaElement):
+
 2016-07-25  Jiewen Tan  <jiewen_tan@apple.com>
 
         Rename SubtleCrypto to WebKitSubtleCrypto
index 8589e99..98ecc9e 100644 (file)
@@ -443,6 +443,7 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
     m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureToAutoplayToExternalDevice);
 #endif
     m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureToControlControlsManager);
+    m_mediaSession->addBehaviorRestriction(MediaElementSession::RequirePlaybackToControlControlsManager);
 
     Settings* settings = document.settings();
 #if PLATFORM(IOS)
index 6e4ee89..283539c 100644 (file)
@@ -1,5 +1,26 @@
 2016-07-25  Wenson Hsieh  <wenson_hsieh@apple.com>
 
+        Media controls should not be displayed for a video until it starts playing
+        https://bugs.webkit.org/show_bug.cgi?id=160092
+        <rdar://problem/26986673>
+
+        Reviewed by Beth Dakin.
+
+        Verify that multiple videos do or don't show the media controller depending on whether videos
+        are playing. Also tweaks an existing API test (VideoControlsManagerSingleLargeVideo) that was
+        passing because we were always showing media controls for large videos with audio, even if they
+        had not played back yet. This change ensures that large videos with audio show media controls
+        only after they begin to play back, and not by virtue of being large enough for main content.
+
+        * TestWebKitAPI/TestWebKitAPI.xcodeproj/project.pbxproj:
+        * TestWebKitAPI/Tests/WebKit2Cocoa/VideoControlsManager.mm:
+        (TestWebKitAPI::TEST):
+        * TestWebKitAPI/Tests/WebKit2Cocoa/large-video-with-audio.html:
+        * TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio-autoplay.html: Added.
+        * TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio.html: Added.
+
+2016-07-25  Wenson Hsieh  <wenson_hsieh@apple.com>
+
         Media controls on apple.com don't disappear when movie finishes playing
         https://bugs.webkit.org/show_bug.cgi?id=160068
         <rdar://problem/26668526>
index 27acf08..2e97864 100644 (file)
@@ -56,6 +56,8 @@
                2E1B7B001D41ABA7007558B4 /* large-video-seek-after-ending.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E1B7AFF1D41A95F007558B4 /* large-video-seek-after-ending.html */; };
                2E1B7B021D41B1B9007558B4 /* large-video-hides-controls-after-seek-to-end.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E1B7B011D41B1B3007558B4 /* large-video-hides-controls-after-seek-to-end.html */; };
                2E1DFDF11D42E1E400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E1DFDF01D42E14400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html */; };
+               2E1DFDED1D42A51100714A00 /* large-videos-with-audio.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E1DFDEC1D42A41C00714A00 /* large-videos-with-audio.html */; };
+               2E1DFDEF1D42A6F200714A00 /* large-videos-with-audio-autoplay.html in Copy Resources */ = {isa = PBXBuildFile; fileRef = 2E1DFDEE1D42A6EB00714A00 /* large-videos-with-audio-autoplay.html */; };
                2E7765CD16C4D80A00BA2BB1 /* mainIOS.mm in Sources */ = {isa = PBXBuildFile; fileRef = 2E7765CC16C4D80A00BA2BB1 /* mainIOS.mm */; };
                2E7765CF16C4D81100BA2BB1 /* mainMac.mm in Sources */ = {isa = PBXBuildFile; fileRef = 2E7765CE16C4D81100BA2BB1 /* mainMac.mm */; };
                33BE5AF9137B5AAE00705813 /* MouseMoveAfterCrash_Bundle.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 33BE5AF8137B5AAE00705813 /* MouseMoveAfterCrash_Bundle.cpp */; };
                                2E1DFDF11D42E1E400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html in Copy Resources */,
                                2E1B7B021D41B1B9007558B4 /* large-video-hides-controls-after-seek-to-end.html in Copy Resources */,
                                2E1B7B001D41ABA7007558B4 /* large-video-seek-after-ending.html in Copy Resources */,
+                               2E1DFDEF1D42A6F200714A00 /* large-videos-with-audio-autoplay.html in Copy Resources */,
+                               2E1DFDED1D42A51100714A00 /* large-videos-with-audio.html in Copy Resources */,
                                5C9E59411D3EB5AC00E3C62E /* ApplicationCache.db in Copy Resources */,
                                5C9E59421D3EB5AC00E3C62E /* ApplicationCache.db-shm in Copy Resources */,
                                5C9E59431D3EB5AC00E3C62E /* ApplicationCache.db-wal in Copy Resources */,
                2E1B7AFF1D41A95F007558B4 /* large-video-seek-after-ending.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-video-seek-after-ending.html"; sourceTree = "<group>"; };
                2E1B7B011D41B1B3007558B4 /* large-video-hides-controls-after-seek-to-end.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-video-hides-controls-after-seek-to-end.html"; sourceTree = "<group>"; };
                2E1DFDF01D42E14400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-video-seek-to-beginning-and-play-after-ending.html"; sourceTree = "<group>"; };
+               2E1DFDEC1D42A41C00714A00 /* large-videos-with-audio.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-with-audio.html"; sourceTree = "<group>"; };
+               2E1DFDEE1D42A6EB00714A00 /* large-videos-with-audio-autoplay.html */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.html; path = "large-videos-with-audio-autoplay.html"; sourceTree = "<group>"; };
                2E7765CC16C4D80A00BA2BB1 /* mainIOS.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = mainIOS.mm; sourceTree = "<group>"; };
                2E7765CE16C4D81100BA2BB1 /* mainMac.mm */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.objcpp; path = mainMac.mm; sourceTree = "<group>"; };
                333B9CE11277F23100FEFCE3 /* PreventEmptyUserAgent.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = PreventEmptyUserAgent.cpp; sourceTree = "<group>"; };
                        children = (
                                2E1DFDF01D42E14400714A00 /* large-video-seek-to-beginning-and-play-after-ending.html */,
                                2E1B7B011D41B1B3007558B4 /* large-video-hides-controls-after-seek-to-end.html */,
+                               2E1DFDEE1D42A6EB00714A00 /* large-videos-with-audio-autoplay.html */,
                                5C9E593E1D3EB1DE00E3C62E /* ApplicationCache.db */,
                                5C9E593F1D3EB1DE00E3C62E /* ApplicationCache.db-shm */,
                                5C9E59401D3EB1DE00E3C62E /* ApplicationCache.db-wal */,
                                51714EB21CF8C761004723C4 /* WebProcessKillIDBCleanup-1.html */,
                                51714EB31CF8C761004723C4 /* WebProcessKillIDBCleanup-2.html */,
                                2E1B7AFF1D41A95F007558B4 /* large-video-seek-after-ending.html */,
+                               2E1DFDEC1D42A41C00714A00 /* large-videos-with-audio.html */,
                        );
                        name = Resources;
                        sourceTree = "<group>";
index e437986..74afc25 100644 (file)
@@ -171,6 +171,54 @@ TEST(VideoControlsManager, VideoControlsManagerSingleSmallVideo)
     TestWebKitAPI::Util::run(&receivedScriptMessage);
 }
 
+TEST(VideoControlsManager, VideoControlsManagerMultipleVideosWithAudio)
+{
+    RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+    configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
+    RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
+
+    __block BOOL didShowMediaControls;
+    __block bool isDoneLoading = false;
+
+    RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
+    [[window contentView] addSubview:webView.get()];
+    RetainPtr<OnLoadMessageHandler> onloadHandler = adoptNS([[OnLoadMessageHandler alloc] initWithWKWebView:webView.get() handler:^() {
+        didShowMediaControls = [webView _hasActiveVideoForControlsManager];
+        isDoneLoading = true;
+    }]);
+    [[configuration userContentController] addScriptMessageHandler:onloadHandler.get() name:@"onloadHandler"];
+
+    NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-videos-with-audio" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
+    [webView loadRequest:request];
+
+    TestWebKitAPI::Util::run(&isDoneLoading);
+    EXPECT_FALSE(didShowMediaControls);
+}
+
+TEST(VideoControlsManager, VideoControlsManagerMultipleVideosWithAudioAndAutoplay)
+{
+    RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
+    configuration.get().mediaTypesRequiringUserActionForPlayback = WKAudiovisualMediaTypeNone;
+    RetainPtr<WKWebView> webView = adoptNS([[WKWebView alloc] initWithFrame:NSMakeRect(0, 0, 100, 100) configuration:configuration.get()]);
+
+    __block BOOL didShowMediaControls;
+    __block bool isDoneLoading = false;
+
+    RetainPtr<NSWindow> window = adoptNS([[NSWindow alloc] initWithContentRect:[webView frame] styleMask:NSBorderlessWindowMask backing:NSBackingStoreBuffered defer:NO]);
+    [[window contentView] addSubview:webView.get()];
+    RetainPtr<OnLoadMessageHandler> onloadHandler = adoptNS([[OnLoadMessageHandler alloc] initWithWKWebView:webView.get() handler:^() {
+        didShowMediaControls = [webView _hasActiveVideoForControlsManager];
+        isDoneLoading = true;
+    }]);
+    [[configuration userContentController] addScriptMessageHandler:onloadHandler.get() name:@"onloadHandler"];
+
+    NSURLRequest *request = [NSURLRequest requestWithURL:[[NSBundle mainBundle] URLForResource:@"large-videos-with-audio-autoplay" withExtension:@"html" subdirectory:@"TestWebKitAPI.resources"]];
+    [webView loadRequest:request];
+
+    TestWebKitAPI::Util::run(&isDoneLoading);
+    EXPECT_TRUE(didShowMediaControls);
+}
+
 TEST(VideoControlsManager, VideoControlsManagerSingleSmallAutoplayingVideo)
 {
     RetainPtr<WKWebViewConfiguration> configuration = adoptNS([[WKWebViewConfiguration alloc] init]);
index 4f1735b..d233463 100644 (file)
 
     function playing() {
         window.clearTimeout(timeout);
-        try {
-            window.webkit.messageHandlers.playingHandler.postMessage('playing');
-        } catch(e) {
-            window.location = 'callback:playing';
-        }
+        setTimeout(function() {
+            try {
+                window.webkit.messageHandlers.playingHandler.postMessage('playing');
+            } catch(e) {
+                window.location = 'callback:playing';
+            }
+        }, 0);
     }
 
     function notPlaying() {
diff --git a/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio-autoplay.html b/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio-autoplay.html
new file mode 100644 (file)
index 0000000..57e8b2d
--- /dev/null
@@ -0,0 +1,18 @@
+<!DOCTYPE html>
+<html>
+<script>
+    function handleLoaded() {
+        // The media controls should be updated on the next runloop.
+        setTimeout(function() {
+            try {
+                window.webkit.messageHandlers.onloadHandler.postMessage("loaded");
+            } catch(e) { }
+        }, 0);
+    }
+</script>
+<body onload=handleLoaded()>
+    <video id="foo" style="width: 480px; height: 320px;"><source src="large-video-with-audio.mp4"></video>
+    <video autoplay id="bar" style="width: 480px; height: 320px;"><source src="large-video-with-audio.mp4"></video>
+    <video id="baz" style="width: 480px; height: 320px;"><source src="large-video-with-audio.mp4"></video>
+</body>
+<html>
diff --git a/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio.html b/Tools/TestWebKitAPI/Tests/WebKit2Cocoa/large-videos-with-audio.html
new file mode 100644 (file)
index 0000000..0fdde2f
--- /dev/null
@@ -0,0 +1,18 @@
+<!DOCTYPE html>
+<html>
+<script>
+    function handleLoaded() {
+        // The media controls should be updated on the next runloop.
+        setTimeout(function() {
+            try {
+                window.webkit.messageHandlers.onloadHandler.postMessage("loaded");
+            } catch(e) { }
+        }, 0);
+    }
+</script>
+<body onload=handleLoaded()>
+    <video id="foo" style="width: 480px; height: 320px;"><source src="large-video-with-audio.mp4"></video>
+    <video id="bar" style="width: 480px; height: 320px;"><source src="large-video-with-audio.mp4"></video>
+    <video id="baz" style="width: 480px; height: 320px;"><source src="large-video-with-audio.mp4"></video>
+</body>
+<html>