[iOS] Disable autoplay of silent videos in low power mode

https://bugs.webkit.org/show_bug.cgi?id=168985
<rdar://problem/30739051>

Reviewed by Jer Noble.

Source/WebCore:

Disable autoplay of silent videos in low power mode on iOS to save battery.
We force the display of the start button when denying autoplay in low power
mode to allow the user to start playback.

Test: media/modern-media-controls/start-support/start-support-lowPowerMode.html

* Modules/mediacontrols/MediaControlsHost.cpp:
(WebCore::MediaControlsHost::shouldForceControlsDisplay):
* Modules/mediacontrols/MediaControlsHost.h:
* Modules/mediacontrols/MediaControlsHost.idl:
Add shouldForceControlsDisplay property on MediaControlsHost. This property
is set to true when we want to force the display for media controls. Currently,
this only returns true for autoplay videos, while in low power mode.

* Modules/modern-media-controls/media/controls-visibility-support.js:
(ControlsVisibilitySupport.prototype._updateControls):
Take into consideration MediaControlsHost.shouldForceControlsDisplay when
initializing shouldShowControls variable.

* Modules/modern-media-controls/media/start-support.js:
(StartSupport.prototype._shouldShowStartButton):
Show the start button when MediaControlsHost.shouldForceControlsDisplay
returns true.

* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::HTMLMediaElement):
Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode restriction
to the session when low power mode is enabled so we know we need to force a
gesture to start playback of autoplay videos.

(WebCore::HTMLMediaElement::shouldForceControlsDisplay):
Add convenience function to decide if we should force display of media controls.
This returns true if the media element is a video with autoplay attribute and
its session has the MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
restriction (i.e. we are in low power mode).

(WebCore::HTMLMediaElement::configureMediaControls):
Force requireControls variable to true if shouldForceControlsDisplay() returns
true. We do this here instead of inside HTMLMediaElement::controls() because
we do not want to change the value of media.controls exposed to JavaScript.

(WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture):
Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode to the list
of restrictions that get removed on user gesture.

* html/MediaElementSession.cpp:
(WebCore::MediaElementSession::playbackPermitted):
Deny playback for videos that have the RequireUserGestureForVideoDueToLowPowerMode
restriction unless there is a user gesture.

* html/MediaElementSession.h:
Add new MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
restriction.

LayoutTests:

Add layout test coverage.

* media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt: Added.
* media/modern-media-controls/start-support/start-support-lowPowerMode.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@213460 268f45cc-cd09-0410-ab3c-d52691b4dbfc

diff --git a/LayoutTests/ChangeLog b/LayoutTests/ChangeLog
index e5639a6..90a54d2 100644 (file)
--- a/LayoutTests/ChangeLog
+++ b/LayoutTests/ChangeLog
@@ -1,3 +1,16 @@
+2017-03-06  Chris Dumez  <cdumez@apple.com>
+
+        [iOS] Disable autoplay of silent videos in low power mode
+        https://bugs.webkit.org/show_bug.cgi?id=168985
+        <rdar://problem/30739051>
+
+        Reviewed by Jer Noble.
+
+        Add layout test coverage.
+
+        * media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt: Added.
+        * media/modern-media-controls/start-support/start-support-lowPowerMode.html: Added.
+
 2017-03-06  Ryan Haddad  <ryanhaddad@apple.com>
 
         Mark media/modern-media-controls/icon-button/icon-button-active-state.html as flaky.

diff --git a/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt
new file mode 100644 (file)
index 0000000..88886c5
--- /dev/null
+++ b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode-expected.txt
@@ -0,0 +1,16 @@
+Test that silent autoplay videos do not start playing without user gesture while in low power mode.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+Received 'canplaythrough' event
+PASS !!internals.shadowRoot(media).querySelector('button.start') became true
+PASS media.controls is false
+Pressing on the start button
+Received 'play' event
+PASS media.controls is false
+PASS internals.shadowRoot(media).querySelector('button.start') became null
+PASS successfullyParsed is true
+
+TEST COMPLETE
+

diff --git a/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode.html b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode.html
new file mode 100644 (file)
index 0000000..6e6b898
--- /dev/null
+++ b/LayoutTests/media/modern-media-controls/start-support/start-support-lowPowerMode.html
@@ -0,0 +1,51 @@
+<!DOCTYPE html>
+<html>
+<head>
+<script>
+internals.setLowPowerModeEnabled(true);
+</script>
+<script src="../../../resources/js-test-pre.js"></script>
+<script src="../resources/media-controls-loader.js" type="text/javascript"></script>
+<script src="../resources/media-controls-utils.js" type="text/javascript"></script>
+</head>
+<body>
+<video src="../../content/test.mp4" style="width: 320px; height: 240px;" autoplay></video>
+<script>
+description("Test that silent autoplay videos do not start playing without user gesture while in low power mode.");
+jsTestIsAsync = true;
+
+let hasUserGesture = false;
+const media = document.querySelector("video");
+
+function endTest()
+{
+    media.remove();
+    finishJSTest();
+}
+
+media.addEventListener("canplaythrough", function() {
+    debug("Received 'canplaythrough' event");
+    // We should display the start button since we denied autoplay and the user needs a way to start playback.
+    shouldBecomeEqual("!!internals.shadowRoot(media).querySelector('button.start')", "true", function() {
+        shouldBeFalse("media.controls");
+
+        debug("Pressing on the start button");
+        hasUserGesture = true;
+        pressOnElement(internals.shadowRoot(media).querySelector('button.start'));
+    });
+});
+
+media.addEventListener("play", function() {
+    debug("Received 'play' event");
+    shouldBeFalse("media.controls");
+    if (hasUserGesture) {
+        shouldBecomeEqual("internals.shadowRoot(media).querySelector('button.start')", "null", endTest);
+    } else {
+        testFailed("Media started playing without user interaction");
+        endTest();
+    }
+});
+</script>
+<script src="../../../resources/js-test-post.js"></script>
+</body>
+</html>

diff --git a/LayoutTests/platform/ios-simulator/TestExpectations b/LayoutTests/platform/ios-simulator/TestExpectations
index f871d06..4c4ed13 100644 (file)
--- a/LayoutTests/platform/ios-simulator/TestExpectations
+++ b/LayoutTests/platform/ios-simulator/TestExpectations
@@ -2827,6 +2827,7 @@ media/modern-media-controls/pip-support/ipad/pip-support-tap.html [ Skip ]
 media/modern-media-controls/playback-support/playback-support-button-click.html [ Skip ]
 media/modern-media-controls/fullscreen-support/ipad/fullscreen-support-tap.html [ Skip ]
 media/modern-media-controls/start-support/start-support-click-to-start.html [ Skip ]
+media/modern-media-controls/start-support/start-support-lowPowerMode.html [ Skip ]
 media/modern-media-controls/button/button.html [ Skip ]
 
 # AirPlay cannot be tested on iOS

diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
index 3d5a44a..5692fa7 100644 (file)
--- a/Source/WebCore/ChangeLog
+++ b/Source/WebCore/ChangeLog
@@ -1,5 +1,67 @@
 2017-03-06  Chris Dumez  <cdumez@apple.com>
 
+        [iOS] Disable autoplay of silent videos in low power mode
+        https://bugs.webkit.org/show_bug.cgi?id=168985
+        <rdar://problem/30739051>
+
+        Reviewed by Jer Noble.
+
+        Disable autoplay of silent videos in low power mode on iOS to save battery.
+        We force the display of the start button when denying autoplay in low power
+        mode to allow the user to start playback.
+
+        Test: media/modern-media-controls/start-support/start-support-lowPowerMode.html
+
+        * Modules/mediacontrols/MediaControlsHost.cpp:
+        (WebCore::MediaControlsHost::shouldForceControlsDisplay):
+        * Modules/mediacontrols/MediaControlsHost.h:
+        * Modules/mediacontrols/MediaControlsHost.idl:
+        Add shouldForceControlsDisplay property on MediaControlsHost. This property
+        is set to true when we want to force the display for media controls. Currently,
+        this only returns true for autoplay videos, while in low power mode.
+
+        * Modules/modern-media-controls/media/controls-visibility-support.js:
+        (ControlsVisibilitySupport.prototype._updateControls):
+        Take into consideration MediaControlsHost.shouldForceControlsDisplay when
+        initializing shouldShowControls variable.
+
+        * Modules/modern-media-controls/media/start-support.js:
+        (StartSupport.prototype._shouldShowStartButton):
+        Show the start button when MediaControlsHost.shouldForceControlsDisplay
+        returns true.
+
+        * html/HTMLMediaElement.cpp:
+        (WebCore::HTMLMediaElement::HTMLMediaElement):
+        Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode restriction
+        to the session when low power mode is enabled so we know we need to force a
+        gesture to start playback of autoplay videos.
+
+        (WebCore::HTMLMediaElement::shouldForceControlsDisplay):
+        Add convenience function to decide if we should force display of media controls.
+        This returns true if the media element is a video with autoplay attribute and
+        its session has the MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
+        restriction (i.e. we are in low power mode).
+
+        (WebCore::HTMLMediaElement::configureMediaControls):
+        Force requireControls variable to true if shouldForceControlsDisplay() returns
+        true. We do this here instead of inside HTMLMediaElement::controls() because
+        we do not want to change the value of media.controls exposed to JavaScript.
+
+        (WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture):
+        Add MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode to the list
+        of restrictions that get removed on user gesture.
+
+        * html/MediaElementSession.cpp:
+        (WebCore::MediaElementSession::playbackPermitted):
+        Deny playback for videos that have the RequireUserGestureForVideoDueToLowPowerMode
+        restriction unless there is a user gesture.
+
+        * html/MediaElementSession.h:
+        Add new MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
+        restriction.
+
+2017-03-06  Chris Dumez  <cdumez@apple.com>
+
         LayoutTest fast/dom/timer-throttling-hidden-page.html is a flaky failure
         https://bugs.webkit.org/show_bug.cgi?id=168927
 

diff --git a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp
index 8d8eeff..c49969e 100644 (file)
--- a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp
+++ b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.cpp
@@ -215,6 +215,11 @@ bool MediaControlsHost::userGestureRequired() const
     return !m_mediaElement->mediaSession().playbackPermitted(*m_mediaElement);
 }
 
+bool MediaControlsHost::shouldForceControlsDisplay() const
+{
+    return m_mediaElement->shouldForceControlsDisplay();
+}
+
 String MediaControlsHost::externalDeviceDisplayName() const
 {
 #if ENABLE(WIRELESS_PLAYBACK_TARGET)

diff --git a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h
index 1f034e5..741f2f7 100644 (file)
--- a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h
+++ b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.h
@@ -70,6 +70,7 @@ public:
     bool isVideoLayerInline() const;
     bool isInMediaDocument() const;
     bool userGestureRequired() const;
+    bool shouldForceControlsDisplay() const;
     void setPreparedToReturnVideoLayerToInline(bool);
 
     void updateCaptionDisplaySizes();

diff --git a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl
index 3e4fcb6..a87bda4 100644 (file)
--- a/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl
+++ b/Source/WebCore/Modules/mediacontrols/MediaControlsHost.idl
@@ -48,6 +48,7 @@ enum DeviceType {
     readonly attribute boolean isVideoLayerInline;
     readonly attribute boolean userGestureRequired;
     readonly attribute boolean isInMediaDocument;
+    readonly attribute boolean shouldForceControlsDisplay;
 
     readonly attribute DOMString externalDeviceDisplayName;
     readonly attribute DeviceType externalDeviceType;

diff --git a/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js b/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js
index e40f9ff..a0de056 100644 (file)
--- a/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js
+++ b/Source/WebCore/Modules/modern-media-controls/media/controls-visibility-support.js
@@ -63,7 +63,8 @@ class ControlsVisibilitySupport extends MediaControllerSupport
     _updateControls()
     {
         const media = this.mediaController.media;
-        const shouldShowControls = !!media.controls;
+        const host = this.mediaController.host;
+        const shouldShowControls = !!(media.controls || (host && host.shouldForceControlsDisplay));
         const isVideo = media instanceof HTMLVideoElement && media.videoTracks.length > 0;
 
         const controls = this.mediaController.controls;

diff --git a/Source/WebCore/Modules/modern-media-controls/media/start-support.js b/Source/WebCore/Modules/modern-media-controls/media/start-support.js
index 8340d2a..40e3cb4 100644 (file)
--- a/Source/WebCore/Modules/modern-media-controls/media/start-support.js
+++ b/Source/WebCore/Modules/modern-media-controls/media/start-support.js
@@ -61,6 +61,10 @@ class StartSupport extends MediaControllerSupport
     _shouldShowStartButton()
     {
         const media = this.mediaController.media;
+        const host = this.mediaController.host;
+
+        if (host && host.shouldForceControlsDisplay)
+            return true;
 
         if (this._hasPlayed || media.played.length)
             return false;
@@ -80,7 +84,6 @@ class StartSupport extends MediaControllerSupport
         if (media.error)
             return false;
 
-        const host = this.mediaController.host;
         if (!media.controls && host && host.allowsInlineMediaPlayback)
             return false;
 

diff --git a/Source/WebCore/html/HTMLMediaElement.cpp b/Source/WebCore/html/HTMLMediaElement.cpp
index 8b9a8c0..be9b2b4 100644 (file)
--- a/Source/WebCore/html/HTMLMediaElement.cpp
+++ b/Source/WebCore/html/HTMLMediaElement.cpp
@@ -456,6 +456,8 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
     m_sendProgressEvents = false;
 #endif
 
+    auto* page = document.page();
+
     if (document.settings().invisibleAutoplayNotPermitted())
         m_mediaSession->addBehaviorRestriction(MediaElementSession::InvisibleAutoplayNotPermitted);
 
@@ -470,6 +472,9 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
                 m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureForLoad);
         }
 
+        if (page && page->isLowPowerModeEnabled())
+            m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode);
+
         if (shouldAudioPlaybackRequireUserGesture)
             m_mediaSession->addBehaviorRestriction(MediaElementSession::RequireUserGestureForAudioRateChange);
 
@@ -493,8 +498,8 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
 #endif
 
 #if ENABLE(VIDEO_TRACK)
-    if (document.page())
-        m_captionDisplayMode = document.page()->group().captionPreferences().captionDisplayMode();
+    if (page)
+        m_captionDisplayMode = page->group().captionPreferences().captionDisplayMode();
 #endif
 
 #if ENABLE(MEDIA_SESSION)
@@ -5931,6 +5936,12 @@ bool HTMLMediaElement::createMediaControls()
 #endif
 }
 
+bool HTMLMediaElement::shouldForceControlsDisplay() const
+{
+    // Always create controls for autoplay video that requires user gesture due to being in low power mode.
+    return isVideo() && autoplay() && m_mediaSession->hasBehaviorRestriction(MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode);
+}
+
 void HTMLMediaElement::configureMediaControls()
 {
     bool requireControls = controls();
@@ -5939,6 +5950,9 @@ void HTMLMediaElement::configureMediaControls()
     if (isVideo() && m_mediaSession->requiresFullscreenForVideoPlayback(*this))
         requireControls = true;
 
+    if (shouldForceControlsDisplay())
+        requireControls = true;
+
     // Always create controls when in full screen mode.
     if (isFullscreen())
         requireControls = true;
@@ -6572,6 +6586,7 @@ void HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture(MediaEle
         | MediaElementSession::RequireUserGestureForVideoRateChange
         | MediaElementSession::RequireUserGestureForAudioRateChange
         | MediaElementSession::RequireUserGestureForFullscreen
+        | MediaElementSession::RequireUserGestureForVideoDueToLowPowerMode
         | MediaElementSession::InvisibleAutoplayNotPermitted
         | MediaElementSession::RequireUserGestureToControlControlsManager);
 

diff --git a/Source/WebCore/html/HTMLMediaElement.h b/Source/WebCore/html/HTMLMediaElement.h
index ccb5e50..e378bc3 100644 (file)
--- a/Source/WebCore/html/HTMLMediaElement.h
+++ b/Source/WebCore/html/HTMLMediaElement.h
@@ -290,6 +290,8 @@ public:
 
     double percentLoaded() const;
 
+    bool shouldForceControlsDisplay() const;
+
 #if ENABLE(VIDEO_TRACK)
     ExceptionOr<TextTrack&> addTextTrack(const String& kind, const String& label, const String& language);
 

diff --git a/Source/WebCore/html/MediaElementSession.cpp b/Source/WebCore/html/MediaElementSession.cpp
index 10ae652..bcec194 100644 (file)
--- a/Source/WebCore/html/MediaElementSession.cpp
+++ b/Source/WebCore/html/MediaElementSession.cpp
@@ -174,6 +174,11 @@ SuccessOr<MediaPlaybackDenialReason> MediaElementSession::playbackPermitted(cons
         return MediaPlaybackDenialReason::UserGestureRequired;
     }
 
+    if (m_restrictions & RequireUserGestureForVideoDueToLowPowerMode && element.isVideo() && !ScriptController::processingUserGestureForMedia()) {
+        LOG(Media, "MediaElementSession::playbackPermitted - returning FALSE because of video low power mode restriction");
+        return MediaPlaybackDenialReason::UserGestureRequired;
+    }
+
     return SuccessOr<MediaPlaybackDenialReason>();
 }
 

diff --git a/Source/WebCore/html/MediaElementSession.h b/Source/WebCore/html/MediaElementSession.h
index 3b6d577..ea38b23 100644 (file)
--- a/Source/WebCore/html/MediaElementSession.h
+++ b/Source/WebCore/html/MediaElementSession.h
@@ -108,6 +108,7 @@ public:
         OverrideUserGestureRequirementForMainContent = 1 << 12,
         RequireUserGestureToControlControlsManager = 1 << 13,
         RequirePlaybackToControlControlsManager = 1 << 14,
+        RequireUserGestureForVideoDueToLowPowerMode = 1 << 15,
         AllRestrictions = ~NoRestrictions,
     };
     typedef unsigned BehaviorRestrictions;