[iOS] Disable autoplay of silent videos in low power mode

https://bugs.webkit.org/show_bug.cgi?id=168985
<rdar://problem/30739051>

Reviewed by Jer Noble.

Source/WebCore:

Disable autoplay of silent videos in low power mode on iOS to save battery.
We force the display of the start button when denying autoplay in low power
mode to allow the user to start playback.

Test: media/modern-media-controls/start-support/start-support-lowPowerMode.html

* Modules/mediacontrols/MediaControlsHost.cpp:
(WebCore::MediaControlsHost::shouldForceControlsDisplay):
* Modules/mediacontrols/MediaControlsHost.h:
* Modules/mediacontrols/MediaControlsHost.idl:
Add shouldForceControlsDisplay property on MediaControlsHost. This property
is set to true when we want to force the display for media controls. Currently,
this only returns true for autoplay videos, while in low power mode.

* Modules/modern-media-controls/media/controls-visibility-support.js:
(ControlsVisibilitySupport.prototype._updateControls):
Take into consideration MediaControlsHost.shouldForceControlsDisplay when
initializing shouldShowControls variable.

* Modules/modern-media-controls/media/start-support.js:
(StartSupport.prototype._shouldShowStartButton):
Show the start button when MediaControlsHost.shouldForceControlsDisplay
returns true.

* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::HTMLMediaElement):
Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode restriction
to the session when low power mode is enabled so we know we need to force a
gesture to start playback of autoplay videos.

(WebCore::HTMLMediaElement::shouldForceControlsDisplay):
Add convenience function to decide if we should force display of media controls.
This returns true if the media element is a video with autoplay attribute and
its session has the MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
restriction (i.e. we are in low power mode).

(WebCore::HTMLMediaElement::configureMediaControls):
Force requireControls variable to true if shouldForceControlsDisplay() returns
true. We do this here instead of inside HTMLMediaElement::controls() because
we do not want to change the value of media.controls exposed to JavaScript.

(WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture):
Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode to the list
of restrictions that get removed on user gesture.

* html/MediaElementSession.cpp:
(WebCore::MediaElementSession::playbackPermitted):
Deny playback for videos that have the RequireUserGestureForVideoDueToLowPowerMode
restriction unless there is a user gesture.

* html/MediaElementSession.h:
Add new MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
restriction.

LayoutTests:

Add layout test coverage.

* media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt: Added.
* media/modern-media-controls/start-support/start-support-lowPowerMode.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213460 268f45cc-cd09-0410-ab3c-d52691b4dbfc

diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog
index e5639a6ebe5409b0c5efd064d6d76d37404bfa7c..90a54d2d8e5112c0ac9acddcc5ff4aad60375e8a 100644 (file)
--- a/LayoutTests/ChangeLog
+++ b/LayoutTests/ChangeLog
@@ -1,3 +1,16 @@
+2017-03-06  Chris Dumez  <cdumez@apple.com>
+
+        [iOS] Disable autoplay of silent videos in low power mode
+        https://bugs.webkit.org/show_bug.cgi?id=168985
+        <rdar://problem/30739051>
+
+        Reviewed by Jer Noble.
+
+        Add layout test coverage.
+
+        * media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt: Added.
+        * media/modern-media-controls/start-support/start-support-lowPowerMode.html: Added.
+
 2017-03-06  Ryan Haddad  <ryanhaddad@apple.com>
 
         Mark media/modern-media-controls/icon-button/icon-button-active-state.html as flaky.

diff --git a/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt
new file mode 100644 (file)
index 0000000..88886c5
--- /dev/null
+++ b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt
@@ -0,0 +1,16 @@
+Test that silent autoplay videos do not start playing without user gesture while in low power mode.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+Received 'canplaythrough' event
+PASS !!internals.shadowRoot(media).querySelector('button.start') became true
+PASS media.controls is false
+Pressing on the start button
+Received 'play' event
+PASS media.controls is false
+PASS internals.shadowRoot(media).querySelector('button.start') became null
+PASS successfullyParsed is true
+
+TEST COMPLETE
+

diff --git a/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode.html b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode.html
new file mode 100644 (file)
index 0000000..6e6b898
--- /dev/null
+++ b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode.html
@@ -0,0 +1,51 @@
+<!DOCTYPE html>
+<html>
+<head>
+<script>
+internals.setLowPowerModeEnabled(true);
+</script>
+<script src="../../../resources/js-test-pre.js"></script>
+<script src="../resources/media-controls-loader.js" type="text/javascript"></script>
+<script src="../resources/media-controls-utils.js" type="text/javascript"></script>
+</head>
+<body>
+<video src="../../content/test.mp4" style="width: 320px; height: 240px;" autoplay></video>
+<script>
+description("Test that silent autoplay videos do not start playing without user gesture while in low power mode.");
+jsTestIsAsync = true;
+
+let hasUserGesture = false;
+const media = document.querySelector("video");
+
+function endTest()
+{
+    media.remove();
+    finishJSTest();
+}
+
+media.addEventListener("canplaythrough", function() {
+    debug("Received 'canplaythrough' event");
+    // We should display the start button since we denied autoplay and the user needs a way to start playback.
+    shouldBecomeEqual("!!internals.shadowRoot(media).querySelector('button.start')", "true", function() {
+        shouldBeFalse("media.controls");
+
+        debug("Pressing on the start button");
+        hasUserGesture = true;
+        pressOnElement(internals.shadowRoot(media).querySelector('button.start'));
+    });
+});
+
+media.addEventListener("play", function() {
+    debug("Received 'play' event");
+    shouldBeFalse("media.controls");
+    if (hasUserGesture) {
+        shouldBecomeEqual("internals.shadowRoot(media).querySelector('button.start')", "null", endTest);
+    } else {
+        testFailed("Media started playing without user interaction");
+        endTest();
+    }
+});
+</script>
+<script src="../../../resources/js-test-post.js"></script>
+</body>
+</html>

diff --git a/LayoutTests/platform/ios-simulator/TestExpectations b/LayoutTests/platform/ios-simulator/TestExpectations
index f871d06f4db5109eda053ac6b4f1eabd3adb2df3..4c4ed13b391050a22cc71e0f3d1549b8c7c70f42 100644 (file)
--- a/LayoutTests/platform/ios-simulator/TestExpectations
+++ b/LayoutTests/platform/ios-simulator/TestExpectations
@@ -2827,6 +2827,7 @@ media/modern-media-controls/pip-support/ipad/pip-support-tap.html [ Skip ]
 media/modern-media-controls/playback-support/playback-support-button-click.html [ Skip ]
 media/modern-media-controls/fullscreen-support/ipad/fullscreen-support-tap.html [ Skip ]
 media/modern-media-controls/start-support/start-support-click-to-start.html [ Skip ]
+media/modern-media-controls/start-support/start-support-lowPowerMode.html [ Skip ]
 media/modern-media-controls/button/button.html [ Skip ]
 
 # AirPlay cannot be tested on iOS

diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
index 3d5a44a4856f881caa816ed52d38cd46e99dd6f5..5692fa7fdf8061734dc2ee8ada8eb4ca5dc6fe19 100644 (file)
--- a/Source/WebCore/ChangeLog
+++ b/Source/WebCore/ChangeLog
@@ -1,3 +1,65 @@
+2017-03-06  Chris Dumez  <cdumez@apple.com>
+
+        [iOS] Disable autoplay of silent videos in low power mode
+        https://bugs.webkit.org/show_bug.cgi?id=168985
+        <rdar://problem/30739051>
+
+        Reviewed by Jer Noble.
+
+        Disable autoplay of silent videos in low power mode on iOS to save battery.
+        We force the display of the start button when denying autoplay in low power
+        mode to allow the user to start playback.
+
+        Test: media/modern-media-controls/start-support/start-support-lowPowerMode.html
+
+        * Modules/mediacontrols/MediaControlsHost.cpp:
+        (WebCore::MediaControlsHost::shouldForceControlsDisplay):
+        * Modules/mediacontrols/MediaControlsHost.h:
+        * Modules/mediacontrols/MediaControlsHost.idl:
+        Add shouldForceControlsDisplay property on MediaControlsHost. This property
+        is set to true when we want to force the display for media controls. Currently,
+        this only returns true for autoplay videos, while in low power mode.
+
+        * Modules/modern-media-controls/media/controls-visibility-support.js:
+        (ControlsVisibilitySupport.prototype._updateControls):
+        Take into consideration MediaControlsHost.shouldForceControlsDisplay when
+        initializing shouldShowControls variable.
+
+        * Modules/modern-media-controls/media/start-support.js:
+        (StartSupport.prototype._shouldShowStartButton):
+        Show the start button when MediaControlsHost.shouldForceControlsDisplay
+        returns true.
+
+        * html/HTMLMediaElement.cpp:
+        (WebCore::HTMLMediaElement::HTMLMediaElement):
+        Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode restriction
+        to the session when low power mode is enabled so we know we need to force a
+        gesture to start playback of autoplay videos.
+
+        (WebCore::HTMLMediaElement::shouldForceControlsDisplay):
+        Add convenience function to decide if we should force display of media controls.
+        This returns true if the media element is a video with autoplay attribute and
+        its session has the MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
+        restriction (i.e. we are in low power mode).
+
+        (WebCore::HTMLMediaElement::configureMediaControls):
+        Force requireControls variable to true if shouldForceControlsDisplay() returns
+        true. We do this here instead of inside HTMLMediaElement::controls() because
+        we do not want to change the value of media.controls exposed to JavaScript.
+
+        (WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture):
+        Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode to the list
+        of restrictions that get removed on user gesture.
+
+        * html/MediaElementSession.cpp:
+        (WebCore::MediaElementSession::playbackPermitted):
+        Deny playback for videos that have the RequireUserGestureForVideoDueToLowPowerMode
+        restriction unless there is a user gesture.
+
+        * html/MediaElementSession.h:
+        Add new MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
+        restriction.
+
 2017-03-06  Chris Dumez  <cdumez@apple.com>
 
         LayoutTest fast/dom/timer-throttling-hidden-page.html is a flaky failure

diff --git a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp
index 8d8eeff0c36f9b119fff28655a1f36e1d8962405..c49969eb0c31965b267cf64a80ed78d55a4941ff 100644 (file)
--- a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp
+++ b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp
@@ -215,6 +215,11 @@ bool MediaControlsHost::userGestureRequired() const
     return !m_mediaElement->mediaSession().playbackPermitted(*m_mediaElement);
 }
 
+bool MediaControlsHost::shouldForceControlsDisplay() const
+{
+    return m_mediaElement->shouldForceControlsDisplay();
+}
+
 String MediaControlsHost::externalDeviceDisplayName() const
 {
 #if ENABLE(WIRELESS_PLAYBACK_TARGET)

diff --git a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h
index 1f034e52275c2835e943f5f1a70dc9de693fdaa9..741f2f758489ea8bbf9ae7f8f36d7941f9040b99 100644 (file)
--- a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h
+++ b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h
@@ -70,6 +70,7 @@ public:
     bool isVideoLayerInline() const;
     bool isInMediaDocument() const;
     bool userGestureRequired() const;
+    bool shouldForceControlsDisplay() const;
     void setPreparedToReturnVideoLayerToInline(bool);
 
     void updateCaptionDisplaySizes();

diff --git a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl
index 3e4fcb6700e5145f6a5bcaf8d2f6a1dcb17db07e..a87bda4e2fa1b7438b12bfcad7de1274b868ad90 100644 (file)
--- a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl
+++ b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl
@@ -48,6 +48,7 @@ enum DeviceType {
     readonly attribute boolean isVideoLayerInline;
     readonly attribute boolean userGestureRequired;
     readonly attribute boolean isInMediaDocument;
+    readonly attribute boolean shouldForceControlsDisplay;
 
     readonly attribute DOMString externalDeviceDisplayName;
     readonly attribute DeviceType externalDeviceType;

diff --git a/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js b/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js
index e40f9ffcb8bb9c52becb3d30ac6894bc2941f960..a0de05686488f5b458d8b43646ef7c4da15f38c8 100644 (file)
--- a/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js
+++ b/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js
@@ -63,7 +63,8 @@ class ControlsVisibilitySupport extends MediaControllerSupport
     _updateControls()
     {
         const media = this.mediaController.media;
-        const shouldShowControls = !!media.controls;
+        const host = this.mediaController.host;
+        const shouldShowControls = !!(media.controls || (host && host.shouldForceControlsDisplay));
         const isVideo = media instanceof HTMLVideoElement && media.videoTracks.length > 0;
 
         const controls = this.mediaController.controls;

diff --git a/Source/WebCore/Modules/modern-media-controls/media/start-support.js b/Source/WebCore/Modules/modern-media-controls/media/start-support.js
index 8340d2a4d16ea763337161dfe4cc4b3119ad6d48..40e3cb43d8156696f77546b75b3e74d737c5a1c1 100644 (file)
--- a/Source/WebCore/Modules/modern-media-controls/media/start-support.js
+++ b/Source/WebCore/Modules/modern-media-controls/media/start-support.js
@@ -61,6 +61,10 @@ class StartSupport extends MediaControllerSupport
     _shouldShowStartButton()
     {
         const media = this.mediaController.media;
+        const host = this.mediaController.host;
+
+        if (host && host.shouldForceControlsDisplay)
+            return true;
 
         if (this._hasPlayed || media.played.length)
             return false;
@@ -80,7 +84,6 @@ class StartSupport extends MediaControllerSupport
         if (media.error)
             return false;
 
-        const host = this.mediaController.host;
         if (!media.controls && host && host.allowsInlineMediaPlayback)
             return false;
 

diff --git a/Source/WebCore/html/HTMLMediaElement.cpp b/Source/WebCore/html/HTMLMediaElement.cpp
index 8b9a8c0bd3b6ca6b4a737e81095bb18e601bec10..be9b2b4a8f9adf51597755aa2d9875520c561f3f 100644 (file)
--- a/Source/WebCore/html/HTMLMediaElement.cpp
+++ b/Source/WebCore/html/HTMLMediaElement.cpp
@@ -456,6 +456,8 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
     m_sendProgressEvents = false;
 #endif
 
+    auto* page = document.page();
+
     if (document.settings().invisibleAutoplayNotPermitted())
         m_mediaSession->addBehaviorRestriction(MediaElementSession::InvisibleAutoplayNotPermitted);
 
@@ -470,6 +472,9 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
                 m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureForLoad);
         }
 
+        if (page && page->isLowPowerModeEnabled())
+            m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode);
+
         if (shouldAudioPlaybackRequireUserGesture)
             m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureForAudioRateChange);
 
@@ -493,8 +498,8 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
 #endif
 
 #if ENABLE(VIDEO_TRACK)
-    if (document.page())
-        m_captionDisplayMode = document.page()->group().captionPreferences().captionDisplayMode();
+    if (page)
+        m_captionDisplayMode = page->group().captionPreferences().captionDisplayMode();
 #endif
 
 #if ENABLE(MEDIA_SESSION)
@@ -5931,6 +5936,12 @@ bool HTMLMediaElement::createMediaControls()
 #endif
 }
 
+bool HTMLMediaElement::shouldForceControlsDisplay() const
+{
+    // Always create controls for autoplay video that requires user gesture due to being in low power mode.
+    return isVideo() && autoplay() && m_mediaSession->hasBehaviorRestriction(MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode);
+}
+
 void HTMLMediaElement::configureMediaControls()
 {
     bool requireControls = controls();
@@ -5939,6 +5950,9 @@ void HTMLMediaElement::configureMediaControls()
     if (isVideo() && m_mediaSession->requiresFullscreenForVideoPlayback(*this))
         requireControls = true;
 
+    if (shouldForceControlsDisplay())
+        requireControls = true;
+
     // Always create controls when in full screen mode.
     if (isFullscreen())
         requireControls = true;
@@ -6572,6 +6586,7 @@ void HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture(MediaEle
         | MediaElementSession::RequireUserGestureForVideoRateChange
         | MediaElementSession::RequireUserGestureForAudioRateChange
         | MediaElementSession::RequireUserGestureForFullscreen
+        | MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
         | MediaElementSession::InvisibleAutoplayNotPermitted
         | MediaElementSession::RequireUserGestureToControlControlsManager);
 

diff --git a/Source/WebCore/html/HTMLMediaElement.h b/Source/WebCore/html/HTMLMediaElement.h
index ccb5e50c440107b1c85fd63968480f4a9c9a17bd..e378bc3fe3c7b7fef4c9c98a7ab54962b14c427d 100644 (file)
--- a/Source/WebCore/html/HTMLMediaElement.h
+++ b/Source/WebCore/html/HTMLMediaElement.h
@@ -290,6 +290,8 @@ public:
 
     double percentLoaded() const;
 
+    bool shouldForceControlsDisplay() const;
+
 #if ENABLE(VIDEO_TRACK)
     ExceptionOr<TextTrack&> addTextTrack(const String& kind, const String& label, const String& language);
 

diff --git a/Source/WebCore/html/MediaElementSession.cpp b/Source/WebCore/html/MediaElementSession.cpp
index 10ae6528767f6ededf5221c04939ec41e57915de..bcec194b6bbe6a4df356c2aa40e2ae1688491fa7 100644 (file)
--- a/Source/WebCore/html/MediaElementSession.cpp
+++ b/Source/WebCore/html/MediaElementSession.cpp
@@ -174,6 +174,11 @@ SuccessOr<MediaPlaybackDenialReason> MediaElementSession::playbackPermitted(cons
         return MediaPlaybackDenialReason::UserGestureRequired;
     }
 
+    if (m_restrictions & RequireUserGestureForVideoDueToLowPowerMode && element.isVideo() && !ScriptController::processingUserGestureForMedia()) {
+        LOG(Media, "MediaElementSession::playbackPermitted - returning FALSE because of video low power mode restriction");
+        return MediaPlaybackDenialReason::UserGestureRequired;
+    }
+
     return SuccessOr<MediaPlaybackDenialReason>();
 }
 

diff --git a/Source/WebCore/html/MediaElementSession.h b/Source/WebCore/html/MediaElementSession.h
index 3b6d577335bf0d964425d098a0c13d91f8520422..ea38b23cab21f9a9d91402dc1db5faf466297b9d 100644 (file)
--- a/Source/WebCore/html/MediaElementSession.h
+++ b/Source/WebCore/html/MediaElementSession.h
@@ -108,6 +108,7 @@ public:
         OverrideUserGestureRequirementForMainContent = 1 << 12,
         RequireUserGestureToControlControlsManager = 1 << 13,
         RequirePlaybackToControlControlsManager = 1 << 14,
+        RequireUserGestureForVideoDueToLowPowerMode = 1 << 15,
         AllRestrictions = ~NoRestrictions,
     };
     typedef unsigned BehaviorRestrictions;