Add a setting and restriction which will pause invisible autoplaying video
authorjer.noble@apple.com <jer.noble@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 2 Dec 2015 22:09:58 +0000 (22:09 +0000)
committerjer.noble@apple.com <jer.noble@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 2 Dec 2015 22:09:58 +0000 (22:09 +0000)
https://bugs.webkit.org/show_bug.cgi?id=151412

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/video-restricted-invisible-autoplay-not-allowed.html

Drive-by fix: m_autoplaying is reset in many places by calling pause() or play(), where those
calls did not originate from an explicit request to pause or play, e.g., during an interruption.
This causes m_autoplaying to be set to false, thus breaking resumption of autoplaying when the
interruption ends. Update PlatformMediaSession to remember its client's "autoplaying" state and
restore it when an interruption ends.

Add a means to register for viewport visibility notifications to FrameView, RenderView,
and RenderElement. Elements who wish to recieve these notifications must do so through their
renderer, and thus will have to re-register whenever a new renderer is attached.

Add a restriction to HTMLMediaElement which will pause autoplaying video when that video scrolls
out of the viewport, or is hidden with CSS.

Add a setting which controls whether that new restriction is set.

* dom/Element.h:
(WebCore::Element::isVisibleInViewportChanged): Add default empty virtual method.
* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::HTMLMediaElement):
(WebCore::HTMLMediaElement::didMoveToNewDocument):
(WebCore::HTMLMediaElement::documentDidResumeFromPageCache):
(WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture):
(WebCore::HTMLMediaElement::resumeAutoplaying):
(WebCore::mediaElementIsAllowedToAutoplay):
(WebCore::HTMLMediaElement::isVisibleInViewportChanged):
(WebCore::HTMLMediaElement::updateShouldAutoplay):
(WebCore::HTMLMediaElement::HTMLMediaElement): Set the new restriction based on the current Settings.
(WebCore::HTMLMediaElement::resumeAutoplaying): Continue autoplay, or begin playback.
(WebCore::HTMLMediaElement::didMoveToNewDocument): Update our autoplay state.
(WebCore::HTMLMediaElement::documentDidResumeFromPageCache): Ditto.
(WebCore::HTMLMediaElement::removedFrom): Ditto.
(WebCore::HTMLMediaElement::didAttachRenderers): Ditto.
(WebCore::HTMLMediaElement::didDetachRenderers): Ditto.
(WebCore::HTMLMediaElement::visibilityDidChange): Ditto.
(WebCore::HTMLMediaElement::willDetachRenderers): Unregister for visibility callbacks.
(WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture): Clear new restriction.
(WebCore::mediaElementIsAllowedToAutoplay): Check for autoplay requirements.
(WebCore::HTMLMediaElement::isVisibleInViewportChanged): Added, update our autoplay state.
(WebCore::HTMLMediaElement::updateShouldAutoplay): Set interruption if necessary, clear otherwise.
* html/HTMLMediaElement.h:
* html/MediaElementSession.cpp:
(WebCore::restrictionName): Added support for new restriction.
* html/MediaElementSession.h:
* page/FrameView.cpp:
(WebCore::FrameView::viewportContentsChanged): Update clients of viewport visibility.
* page/Settings.in:
* platform/audio/PlatformMediaSession.cpp:
(WebCore::stateName): Add new "Autoplay" state.
(WebCore::interruptionName): Added new interruption type.
(WebCore::PlatformMediaSession::beginInterruption): Set the m_interruptionType.
(WebCore::PlatformMediaSession::clientWillBeginAutoplaying): Set the m_state to Autoplaying.
* platform/audio/PlatformMediaSession.h:
(WebCore::PlatformMediaSession::interruptionType): Added getter.
(WebCore::PlatformMediaSessionClient::resumeAutoplaying): Added default.
* platform/audio/PlatformMediaSessionManager.cpp:
(WebCore::PlatformMediaSessionManager::sessionWillBeginPlayback): Only pause session if its state is playing.
* rendering/RenderElement.cpp:
(WebCore::RenderElement::RenderElement): Set new ivars.
(WebCore::RenderElement::~RenderElement): Unregister for callbacks if necessary.
(WebCore::RenderElement::registerForVisibleInViewportCallback): Register for callbacks from RenderView.
(WebCore::RenderElement::unregisterForVisibleInViewportCallback): Unregister from same.
(WebCore::RenderElement::visibleInViewportStateChanged): Notify Element if value has changed.
* rendering/RenderElement.h:
* rendering/RenderView.cpp:
(WebCore::RenderView::registerForVisibleInViewportCallback): Add renderer to list of callbacks.
(WebCore::RenderView::unregisterForVisibleInViewportCallback): Remove renderer from same.
(WebCore::RenderView::updateVisibleViewportRect): Walk renderers setting their visiblility based on the viewport visible rect.
* rendering/RenderView.h:
* testing/Internals.cpp:
(WebCore::Internals::setMediaElementRestrictions): Support new restriction.

LayoutTests:

* media/video-restricted-invisible-autoplay-not-allowed-expected.txt: Added.
* media/video-restricted-invisible-autoplay-not-allowed.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@192966 268f45cc-cd09-0410-ab3c-d52691b4dbfc

24 files changed:
LayoutTests/ChangeLog
LayoutTests/media/video-restricted-invisible-autoplay-not-allowed-expected.txt [new file with mode: 0644]
LayoutTests/media/video-restricted-invisible-autoplay-not-allowed.html [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/dom/Element.h
Source/WebCore/html/HTMLMediaElement.cpp
Source/WebCore/html/HTMLMediaElement.h
Source/WebCore/html/MediaElementSession.cpp
Source/WebCore/html/MediaElementSession.h
Source/WebCore/page/FrameView.cpp
Source/WebCore/page/Settings.in
Source/WebCore/platform/audio/PlatformMediaSession.cpp
Source/WebCore/platform/audio/PlatformMediaSession.h
Source/WebCore/platform/audio/PlatformMediaSessionManager.cpp
Source/WebCore/rendering/RenderElement.cpp
Source/WebCore/rendering/RenderElement.h
Source/WebCore/rendering/RenderImage.h
Source/WebCore/rendering/RenderMedia.cpp
Source/WebCore/rendering/RenderMedia.h
Source/WebCore/rendering/RenderObject.cpp
Source/WebCore/rendering/RenderObject.h
Source/WebCore/rendering/RenderView.cpp
Source/WebCore/rendering/RenderView.h
Source/WebCore/testing/Internals.cpp

index d67b6a0..4eefc30 100644 (file)
@@ -1,3 +1,13 @@
+2015-12-02  Jer Noble  <jer.noble@apple.com>
+
+        Add a setting and restriction which will pause invisible autoplaying video
+        https://bugs.webkit.org/show_bug.cgi?id=151412
+
+        Reviewed by Eric Carlson.
+
+        * media/video-restricted-invisible-autoplay-not-allowed-expected.txt: Added.
+        * media/video-restricted-invisible-autoplay-not-allowed.html: Added.
+
 2015-12-02  Ryan Haddad  <ryanhaddad@apple.com>
 
         Rebaseline fast/parser/xml-declaration-missing-ending-mark.html, fast/parser/xml-colon-entity.html for Win
diff --git a/LayoutTests/media/video-restricted-invisible-autoplay-not-allowed-expected.txt b/LayoutTests/media/video-restricted-invisible-autoplay-not-allowed-expected.txt
new file mode 100644 (file)
index 0000000..8639df3
--- /dev/null
@@ -0,0 +1,22 @@
+
+Test that "invisible autoplay not allowed restriction" pauses media when scrolled out of view.
+
+** setting video.src
+RUN(internals.setMediaElementRestrictions(video, "InvisibleAutoplayNotPermitted"))
+
+EVENT(play)
+RUN(video.style.display = "none")
+EVENT(pause)
+RUN(video.style.removeProperty("display"))
+EVENT(play)
+RUN(video.style.visibility = "hidden")
+EVENT(pause)
+RUN(video.style.removeProperty("visibility"))
+EVENT(play)
+RUN(document.documentElement.style.height = window.innerHeight + 20 + video.offsetHeight)
+RUN(window.scrollBy(0, 20 + video.offsetHeight))
+EVENT(pause)
+RUN(window.scrollTo(0, 0))
+EVENT(play)
+END OF TEST
+
diff --git a/LayoutTests/media/video-restricted-invisible-autoplay-not-allowed.html b/LayoutTests/media/video-restricted-invisible-autoplay-not-allowed.html
new file mode 100644 (file)
index 0000000..b087a13
--- /dev/null
@@ -0,0 +1,60 @@
+<html>
+    <head>
+        <script src="media-file.js"></script>
+        <script src="video-test.js"></script>
+        <script>
+            function start()
+            {
+                findMediaElement();
+                consoleWrite('** setting video.src');
+                video.src = findMediaFile('video', 'content/test');
+
+                waitForEventOnce('play', play1);
+                run('internals.setMediaElementRestrictions(video, "InvisibleAutoplayNotPermitted")');
+                consoleWrite('');
+            }
+
+            function play1()
+            {
+                run('video.style.display = "none"');
+                waitForEventOnce('pause', pause1);
+            }
+
+            function pause1()
+            {
+                run('video.style.removeProperty("display")')
+                waitForEventOnce('play', play2);
+            }
+
+            function play2()
+            {
+                run('video.style.visibility = "hidden"');
+                waitForEventOnce('pause', pause2);
+            }
+
+            function pause2()
+            {
+                run('video.style.removeProperty("visibility")');
+                waitForEventOnce('play', play3);
+            }
+
+            function play3()
+            {
+                run('document.documentElement.style.height = window.innerHeight + 20 + video.offsetHeight');
+                run('window.scrollBy(0, 20 + video.offsetHeight)');
+                waitForEventOnce('pause', pause3);
+            }
+
+            function pause3()
+            {
+                run('window.scrollTo(0, 0)');
+                waitForEventOnce('play', endTest);
+            }
+        </script>
+    </head>
+
+    <body onload="start()">
+        <video controls autoplay></video>
+        <p>Test that "invisible autoplay not allowed restriction" pauses media when scrolled out of view.</p>
+    </body>
+</html>
index 3d932b9..d58596f 100644 (file)
@@ -1,3 +1,83 @@
+2015-12-02  Jer Noble  <jer.noble@apple.com>
+
+        Add a setting and restriction which will pause invisible autoplaying video
+        https://bugs.webkit.org/show_bug.cgi?id=151412
+
+        Reviewed by Eric Carlson.
+
+        Test: media/video-restricted-invisible-autoplay-not-allowed.html
+
+        Drive-by fix: m_autoplaying is reset in many places by calling pause() or play(), where those
+        calls did not originate from an explicit request to pause or play, e.g., during an interruption.
+        This causes m_autoplaying to be set to false, thus breaking resumption of autoplaying when the
+        interruption ends. Update PlatformMediaSession to remember its client's "autoplaying" state and
+        restore it when an interruption ends.
+
+        Add a means to register for viewport visibility notifications to FrameView, RenderView,
+        and RenderElement. Elements who wish to recieve these notifications must do so through their
+        renderer, and thus will have to re-register whenever a new renderer is attached.
+
+        Add a restriction to HTMLMediaElement which will pause autoplaying video when that video scrolls
+        out of the viewport, or is hidden with CSS.
+
+        Add a setting which controls whether that new restriction is set.
+
+        * dom/Element.h:
+        (WebCore::Element::isVisibleInViewportChanged): Add default empty virtual method.
+        * html/HTMLMediaElement.cpp:
+        (WebCore::HTMLMediaElement::HTMLMediaElement):
+        (WebCore::HTMLMediaElement::didMoveToNewDocument):
+        (WebCore::HTMLMediaElement::documentDidResumeFromPageCache):
+        (WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture):
+        (WebCore::HTMLMediaElement::resumeAutoplaying):
+        (WebCore::mediaElementIsAllowedToAutoplay):
+        (WebCore::HTMLMediaElement::isVisibleInViewportChanged):
+        (WebCore::HTMLMediaElement::updateShouldAutoplay):
+        (WebCore::HTMLMediaElement::HTMLMediaElement): Set the new restriction based on the current Settings.
+        (WebCore::HTMLMediaElement::resumeAutoplaying): Continue autoplay, or begin playback.
+        (WebCore::HTMLMediaElement::didMoveToNewDocument): Update our autoplay state.
+        (WebCore::HTMLMediaElement::documentDidResumeFromPageCache): Ditto.
+        (WebCore::HTMLMediaElement::removedFrom): Ditto.
+        (WebCore::HTMLMediaElement::didAttachRenderers): Ditto.
+        (WebCore::HTMLMediaElement::didDetachRenderers): Ditto.
+        (WebCore::HTMLMediaElement::visibilityDidChange): Ditto.
+        (WebCore::HTMLMediaElement::willDetachRenderers): Unregister for visibility callbacks.
+        (WebCore::HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture): Clear new restriction.
+        (WebCore::mediaElementIsAllowedToAutoplay): Check for autoplay requirements.
+        (WebCore::HTMLMediaElement::isVisibleInViewportChanged): Added, update our autoplay state.
+        (WebCore::HTMLMediaElement::updateShouldAutoplay): Set interruption if necessary, clear otherwise.
+        * html/HTMLMediaElement.h:
+        * html/MediaElementSession.cpp:
+        (WebCore::restrictionName): Added support for new restriction.
+        * html/MediaElementSession.h:
+        * page/FrameView.cpp:
+        (WebCore::FrameView::viewportContentsChanged): Update clients of viewport visibility.
+        * page/Settings.in:
+        * platform/audio/PlatformMediaSession.cpp:
+        (WebCore::stateName): Add new "Autoplay" state.
+        (WebCore::interruptionName): Added new interruption type.
+        (WebCore::PlatformMediaSession::beginInterruption): Set the m_interruptionType.
+        (WebCore::PlatformMediaSession::clientWillBeginAutoplaying): Set the m_state to Autoplaying.
+        * platform/audio/PlatformMediaSession.h:
+        (WebCore::PlatformMediaSession::interruptionType): Added getter.
+        (WebCore::PlatformMediaSessionClient::resumeAutoplaying): Added default.
+        * platform/audio/PlatformMediaSessionManager.cpp:
+        (WebCore::PlatformMediaSessionManager::sessionWillBeginPlayback): Only pause session if its state is playing.
+        * rendering/RenderElement.cpp:
+        (WebCore::RenderElement::RenderElement): Set new ivars.
+        (WebCore::RenderElement::~RenderElement): Unregister for callbacks if necessary.
+        (WebCore::RenderElement::registerForVisibleInViewportCallback): Register for callbacks from RenderView.
+        (WebCore::RenderElement::unregisterForVisibleInViewportCallback): Unregister from same.
+        (WebCore::RenderElement::visibleInViewportStateChanged): Notify Element if value has changed.
+        * rendering/RenderElement.h:
+        * rendering/RenderView.cpp:
+        (WebCore::RenderView::registerForVisibleInViewportCallback): Add renderer to list of callbacks.
+        (WebCore::RenderView::unregisterForVisibleInViewportCallback): Remove renderer from same.
+        (WebCore::RenderView::updateVisibleViewportRect): Walk renderers setting their visiblility based on the viewport visible rect.
+        * rendering/RenderView.h:
+        * testing/Internals.cpp:
+        (WebCore::Internals::setMediaElementRestrictions): Support new restriction.
+
 2015-12-02  Brady Eidson  <beidson@apple.com>
 
         Modern IDB: IDBTransaction::error is not exposed.
index 24180d1..1f14861 100644 (file)
@@ -494,6 +494,8 @@ public:
     StyleResolver& styleResolver();
     Ref<RenderStyle> resolveStyle(RenderStyle* parentStyle);
 
+    virtual void isVisibleInViewportChanged() { }
+
 protected:
     Element(const QualifiedName&, Document&, ConstructionType);
 
index e0cdb89..5523726 100644 (file)
@@ -427,6 +427,8 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document& docum
         // Relax RequireUserGestureForFullscreen when requiresUserGestureForMediaPlayback is not set:
         m_mediaSession->removeBehaviorRestriction(MediaElementSession::RequireUserGestureForFullscreen);
     }
+    if (settings && settings->invisibleAutoplayNotPermitted())
+        m_mediaSession->addBehaviorRestriction(MediaElementSession::InvisibleAutoplayNotPermitted);
 #endif // !PLATFORM(IOS)
 
     if (settings && settings->audioPlaybackRequiresUserGesture() && settings->requiresUserGestureForMediaPlayback())
@@ -602,6 +604,7 @@ void HTMLMediaElement::didMoveToNewDocument(Document* oldDocument)
     registerWithDocument(document());
 
     HTMLElement::didMoveToNewDocument(oldDocument);
+    updateShouldAutoplay();
 }
 
 #if ENABLE(WIRELESS_PLAYBACK_TARGET)
@@ -613,6 +616,7 @@ void HTMLMediaElement::prepareForDocumentSuspension()
 void HTMLMediaElement::resumeFromDocumentSuspension()
 {
     m_mediaSession->registerWithDocument(document());
+    updateShouldAutoplay();
 }
 #endif
 
@@ -779,8 +783,23 @@ void HTMLMediaElement::willAttachRenderers()
 
 void HTMLMediaElement::didAttachRenderers()
 {
-    if (renderer())
-        renderer()->updateFromElement();
+    if (RenderElement* renderer = this->renderer()) {
+        renderer->updateFromElement();
+        if (m_mediaSession->hasBehaviorRestriction(MediaElementSession::InvisibleAutoplayNotPermitted))
+            renderer->registerForVisibleInViewportCallback();
+    }
+    updateShouldAutoplay();
+}
+
+void HTMLMediaElement::willDetachRenderers()
+{
+    if (renderer() && m_mediaSession->hasBehaviorRestriction(MediaElementSession::InvisibleAutoplayNotPermitted))
+        renderer()->unregisterForVisibleInViewportCallback();
+}
+
+void HTMLMediaElement::didDetachRenderers()
+{
+    updateShouldAutoplay();
 }
 
 void HTMLMediaElement::didRecalcStyle(Style::Change)
@@ -2058,6 +2077,15 @@ void HTMLMediaElement::mediaPlayerReadyStateChanged(MediaPlayer*)
     endProcessingMediaPlayerCallback();
 }
 
+static bool elementCanTransitionFromAutoplayToPlay(HTMLMediaElement& element)
+{
+    return element.isAutoplaying()
+        && element.paused()
+        && element.autoplay()
+        && !element.document().isSandboxed(SandboxAutomaticFeatures)
+        && element.mediaSession().playbackPermitted(element);
+}
+
 void HTMLMediaElement::setReadyState(MediaPlayer::ReadyState state)
 {
     LOG(Media, "HTMLMediaElement::setReadyState(%p) - new state = %d, current state = %d,", this, static_cast<int>(state), static_cast<int>(m_readyState));
@@ -2167,7 +2195,7 @@ void HTMLMediaElement::setReadyState(MediaPlayer::ReadyState state)
         if (isPotentiallyPlaying && oldState <= HAVE_CURRENT_DATA)
             scheduleEvent(eventNames().playingEvent);
 
-        if (m_autoplaying && m_paused && autoplay() && !document().isSandboxed(SandboxAutomaticFeatures) && m_mediaSession->playbackPermitted(*this)) {
+        if (elementCanTransitionFromAutoplayToPlay(*this)) {
             m_paused = false;
             invalidateCachedTime();
             scheduleEvent(eventNames().playEvent);
@@ -3942,6 +3970,11 @@ void HTMLMediaElement::layoutSizeChanged()
 #endif
 }
 
+void HTMLMediaElement::visibilityDidChange()
+{
+    updateShouldAutoplay();
+}
+
 void HTMLMediaElement::setSelectedTextTrack(TextTrack* trackToSelect)
 {
     TextTrackList* trackList = textTracks();
@@ -6174,7 +6207,8 @@ void HTMLMediaElement::removeBehaviorsRestrictionsAfterFirstUserGesture()
         | MediaElementSession::RequireUserGestureForLoad
         | MediaElementSession::RequireUserGestureForRateChange
         | MediaElementSession::RequireUserGestureForAudioRateChange
-        | MediaElementSession::RequireUserGestureForFullscreen;
+        | MediaElementSession::RequireUserGestureForFullscreen
+        | MediaElementSession::InvisibleAutoplayNotPermitted;
     m_mediaSession->removeBehaviorRestriction(restrictionsToRemove);
 }
 
@@ -6455,6 +6489,15 @@ void HTMLMediaElement::suspendPlayback()
         pause();
 }
 
+void HTMLMediaElement::resumeAutoplaying()
+{
+    LOG(Media, "HTMLMediaElement::resumeAutoplaying(%p) - paused = %s", this, boolString(paused()));
+    m_autoplaying = true;
+
+    if (elementCanTransitionFromAutoplayToPlay(*this))
+        play();
+}
+
 void HTMLMediaElement::mayResumePlayback(bool shouldResume)
 {
     LOG(Media, "HTMLMediaElement::mayResumePlayback(%p) - paused = %s", this, boolString(paused()));
@@ -6694,6 +6737,49 @@ void HTMLMediaElement::allowsMediaDocumentInlinePlaybackChanged()
         enterFullscreen();
 }
 
+static bool mediaElementIsAllowedToAutoplay(const HTMLMediaElement& element)
+{
+    const Document& document = element.document();
+    if (document.inPageCache())
+        return false;
+    if (document.activeDOMObjectsAreSuspended())
+        return false;
+
+    RenderElement* renderer = element.renderer();
+    if (!renderer)
+        return false;
+    if (renderer->style().visibility() != VISIBLE)
+        return false;
+    if (renderer->view().frameView().isOffscreen())
+        return false;
+    if (renderer->visibleInViewportState() != RenderElement::VisibleInViewport)
+        return false;
+    return true;
+}
+
+void HTMLMediaElement::isVisibleInViewportChanged()
+{
+    updateShouldAutoplay();
+}
+
+void HTMLMediaElement::updateShouldAutoplay()
+{
+    if (!autoplay())
+        return;
+
+    if (!m_mediaSession->hasBehaviorRestriction(MediaElementSession::InvisibleAutoplayNotPermitted))
+        return;
+
+    bool canAutoplay = mediaElementIsAllowedToAutoplay(*this);
+    if (canAutoplay
+        && m_mediaSession->state() == PlatformMediaSession::Interrupted
+        && m_mediaSession->interruptionType() == PlatformMediaSession::InvisibleAutoplay)
+        m_mediaSession->endInterruption(PlatformMediaSession::MayResumePlaying);
+    else if (!canAutoplay
+        && m_mediaSession->state() != PlatformMediaSession::Interrupted)
+        m_mediaSession->beginInterruption(PlatformMediaSession::InvisibleAutoplay);
+}
+
 }
 
 #endif
index d4cc796..307eacb 100644 (file)
@@ -200,7 +200,8 @@ public:
     virtual PassRefPtr<TimeRanges> seekable() const override;
     WEBCORE_EXPORT bool ended() const;
     bool autoplay() const;
-    bool loop() const;    
+    bool isAutoplaying() const { return m_autoplaying; }
+    bool loop() const;
     void setLoop(bool b);
     WEBCORE_EXPORT virtual void play() override;
     WEBCORE_EXPORT virtual void pause() override;
@@ -446,6 +447,7 @@ public:
     virtual MediaProducer::MediaStateFlags mediaState() const override;
 
     void layoutSizeChanged();
+    void visibilityDidChange();
 
     void allowsMediaDocumentInlinePlaybackChanged();
 
@@ -453,13 +455,15 @@ protected:
     HTMLMediaElement(const QualifiedName&, Document&, bool);
     virtual ~HTMLMediaElement();
 
-    virtual void parseAttribute(const QualifiedName&, const AtomicString&) override;
-    virtual void finishParsingChildren() override;
-    virtual bool isURLAttribute(const Attribute&) const override;
-    virtual void willAttachRenderers() override;
-    virtual void didAttachRenderers() override;
+    void parseAttribute(const QualifiedName&, const AtomicString&) override;
+    void finishParsingChildren() override;
+    bool isURLAttribute(const Attribute&) const override;
+    void willAttachRenderers() override;
+    void didAttachRenderers() override;
+    void willDetachRenderers() override;
+    void didDetachRenderers() override;
 
-    virtual void didMoveToNewDocument(Document* oldDocument) override;
+    void didMoveToNewDocument(Document* oldDocument) override;
 
     enum DisplayMode { Unknown, None, Poster, PosterWaitingForVideo, Video };
     DisplayMode displayMode() const { return m_displayMode; }
@@ -713,7 +717,6 @@ private:
     bool isBlockedOnMediaController() const;
     virtual bool hasCurrentSrc() const override { return !m_currentSrc.isEmpty(); }
     virtual bool isLiveStream() const override { return movieLoadType() == MediaPlayerEnums::LiveStream; }
-    bool isAutoplaying() const { return m_autoplaying; }
 
     void updateSleepDisabling();
     bool shouldDisableSleep() const;
@@ -729,6 +732,7 @@ private:
     PlatformMediaSession::MediaType presentationType() const override;
     PlatformMediaSession::DisplayType displayType() const override;
     void suspendPlayback() override;
+    void resumeAutoplaying() override;
     void mayResumePlayback(bool shouldResume) override;
     String mediaSessionTitle() const override;
     double mediaSessionDuration() const override { return duration(); }
@@ -757,6 +761,9 @@ private:
     void updateMediaState(UpdateMediaState updateState = UpdateMediaState::Synchronously);
 #endif
 
+    void isVisibleInViewportChanged() override final;
+    void updateShouldAutoplay();
+
     Timer m_pendingActionTimer;
     Timer m_progressEventTimer;
     Timer m_playbackProgressTimer;
index 8f6e4d8..f4e8f40 100644 (file)
@@ -73,6 +73,7 @@ static String restrictionName(MediaElementSession::BehaviorRestrictions restrict
     CASE(WirelessVideoPlaybackDisabled);
 #endif
     CASE(RequireUserGestureForAudioRateChange);
+    CASE(InvisibleAutoplayNotPermitted);
 
     return restrictionBuilder.toString();
 }
index 0e5af7a..351cb0e 100644 (file)
@@ -91,6 +91,7 @@ public:
 #endif
         MetadataPreloadingNotPermitted = 1 << 9,
         AutoPreloadingNotPermitted = 1 << 10,
+        InvisibleAutoplayNotPermitted = 1 << 11,
     };
     typedef unsigned BehaviorRestrictions;
 
index 31ac31e..1387121 100644 (file)
@@ -1887,6 +1887,9 @@ void FrameView::viewportContentsChanged()
     applyRecursivelyWithVisibleRect([] (FrameView& frameView, const IntRect& visibleRect) {
         frameView.resumeVisibleImageAnimations(visibleRect);
         frameView.updateScriptedAnimationsAndTimersThrottlingState(visibleRect);
+
+        if (auto* renderView = frameView.frame().contentRenderer())
+            renderView->updateVisibleViewportRect(visibleRect);
     });
 }
 
index 8b4cc2d..b957060 100644 (file)
@@ -130,6 +130,7 @@ allowsInlineMediaPlayback initial=defaultAllowsInlineMediaPlayback
 inlineMediaPlaybackRequiresPlaysInlineAttribute initial=defaultInlineMediaPlaybackRequiresPlaysInlineAttribute
 allowsPictureInPictureMediaPlayback initial=defaultAllowsPictureInPictureMediaPlayback
 mediaControlsScaleWithPageZoom initial=defaultMediaControlsScaleWithPageZoom
+invisibleAutoplayNotPermitted initial=false
 passwordEchoEnabled initial=false
 suppressesIncrementalRendering initial=false
 incrementalRenderingSuppressionTimeoutInSeconds type=double, initial=defaultIncrementalRenderingSuppressionTimeoutInSeconds
index afd948c..5e261d6 100644 (file)
@@ -42,6 +42,7 @@ static const char* stateName(PlatformMediaSession::State state)
 #define STATE_CASE(state) case PlatformMediaSession::state: return #state
     switch (state) {
     STATE_CASE(Idle);
+    STATE_CASE(Autoplaying);
     STATE_CASE(Playing);
     STATE_CASE(Paused);
     STATE_CASE(Interrupted);
@@ -55,10 +56,12 @@ static const char* interruptionName(PlatformMediaSession::InterruptionType type)
 {
 #define INTERRUPTION_CASE(type) case PlatformMediaSession::type: return #type
     switch (type) {
+    INTERRUPTION_CASE(NoInterruption);
     INTERRUPTION_CASE(SystemSleep);
     INTERRUPTION_CASE(EnteringBackground);
     INTERRUPTION_CASE(SystemInterruption);
     INTERRUPTION_CASE(SuspendedUnderLock);
+    INTERRUPTION_CASE(InvisibleAutoplay);
     }
     
     ASSERT_NOT_REACHED();
@@ -106,6 +109,7 @@ void PlatformMediaSession::beginInterruption(InterruptionType type)
     m_stateToRestore = state();
     m_notifyingClient = true;
     setState(Interrupted);
+    m_interruptionType = type;
     client().suspendPlayback();
     m_notifyingClient = false;
 }
@@ -124,12 +128,32 @@ void PlatformMediaSession::endInterruption(EndInterruptionFlags flags)
 
     State stateToRestore = m_stateToRestore;
     m_stateToRestore = Idle;
+    m_interruptionType = NoInterruption;
     setState(Paused);
 
+    if (stateToRestore == Autoplaying)
+        client().resumeAutoplaying();
+
     bool shouldResume = flags & MayResumePlaying && stateToRestore == Playing;
     client().mayResumePlayback(shouldResume);
 }
 
+void PlatformMediaSession::clientWillBeginAutoplaying()
+{
+    if (m_notifyingClient)
+        return;
+
+    LOG(Media, "PlatformMediaSession::clientWillBeginAutoplaying(%p)- state = %s", this, stateName(m_state));
+    if (state() == Interrupted) {
+        m_stateToRestore = Autoplaying;
+        LOG(Media, "      setting stateToRestore to \"Autoplaying\"");
+        return;
+    }
+
+    setState(Autoplaying);
+    updateClientDataBuffering();
+}
+
 bool PlatformMediaSession::clientWillBeginPlayback()
 {
     if (m_notifyingClient)
index 13bcefd..69acbf6 100644 (file)
@@ -63,6 +63,7 @@ public:
 
     enum State {
         Idle,
+        Autoplaying,
         Playing,
         Paused,
         Interrupted,
@@ -71,11 +72,15 @@ public:
     void setState(State);
 
     enum InterruptionType {
+        NoInterruption,
         SystemSleep,
         EnteringBackground,
         SystemInterruption,
         SuspendedUnderLock,
+        InvisibleAutoplay,
     };
+    InterruptionType interruptionType() const { return m_interruptionType; }
+
     enum EndInterruptionFlags {
         NoFlags = 0,
         MayResumePlaying = 1 << 0,
@@ -84,6 +89,7 @@ public:
     void beginInterruption(InterruptionType);
     void endInterruption(EndInterruptionFlags);
 
+    void clientWillBeginAutoplaying();
     bool clientWillBeginPlayback();
     bool clientWillPausePlayback();
 
@@ -148,6 +154,7 @@ private:
     Timer m_clientDataBufferingTimer;
     State m_state;
     State m_stateToRestore;
+    InterruptionType m_interruptionType { NoInterruption };
     int m_interruptionCount { 0 };
     bool m_notifyingClient;
     bool m_isPlayingToWirelessPlaybackTarget { false };
@@ -165,6 +172,7 @@ public:
     virtual PlatformMediaSession::MediaType presentationType() const = 0;
     virtual PlatformMediaSession::DisplayType displayType() const { return PlatformMediaSession::Normal; }
 
+    virtual void resumeAutoplaying() { }
     virtual void mayResumePlayback(bool shouldResume) = 0;
     virtual void suspendPlayback() = 0;
 
index e3f62de..dd9ff30 100644 (file)
@@ -208,7 +208,9 @@ bool PlatformMediaSessionManager::sessionWillBeginPlayback(PlatformMediaSession&
     for (auto* oneSession : sessions) {
         if (oneSession == &session)
             continue;
-        if (oneSession->mediaType() == sessionType && restrictions & ConcurrentPlaybackNotPermitted)
+        if (oneSession->mediaType() == sessionType
+            && restrictions & ConcurrentPlaybackNotPermitted
+            && oneSession->state() == PlatformMediaSession::Playing)
             oneSession->pauseSession();
     }
 
index 8c2ea08..5d882b8 100644 (file)
@@ -141,6 +141,8 @@ RenderElement::~RenderElement()
     }
     if (m_hasPausedImageAnimations)
         view().removeRendererWithPausedImageAnimations(*this);
+    if (isRegisteredForVisibleInViewportCallback())
+        view().unregisterForVisibleInViewportCallback(*this);
 }
 
 RenderPtr<RenderElement> RenderElement::createFor(Element& element, Ref<RenderStyle>&& style)
@@ -1464,6 +1466,35 @@ static bool shouldRepaintForImageAnimation(const RenderElement& renderer, const
     return true;
 }
 
+void RenderElement::registerForVisibleInViewportCallback()
+{
+    if (isRegisteredForVisibleInViewportCallback())
+        return;
+    setIsRegisteredForVisibleInViewportCallback(true);
+
+    view().registerForVisibleInViewportCallback(*this);
+}
+
+void RenderElement::unregisterForVisibleInViewportCallback()
+{
+    if (!isRegisteredForVisibleInViewportCallback())
+        return;
+    setIsRegisteredForVisibleInViewportCallback(false);
+
+    view().unregisterForVisibleInViewportCallback(*this);
+    m_visibleInViewportState = VisibilityUnknown;
+}
+
+void RenderElement::visibleInViewportStateChanged(VisibleInViewportState state)
+{
+    if (state == visibleInViewportState())
+        return;
+    setVisibleInViewportState(state);
+
+    if (element())
+        element()->isVisibleInViewportChanged();
+}
+
 void RenderElement::newImageAnimationFrameAvailable(CachedImage& image)
 {
     if (document().inPageCache())
index 6b1d6c4..802123f 100644 (file)
@@ -193,6 +193,10 @@ public:
     bool hasShapeOutside() const { return false; }
 #endif
 
+    void registerForVisibleInViewportCallback();
+    void unregisterForVisibleInViewportCallback();
+    void visibleInViewportStateChanged(VisibleInViewportState);
+
     bool repaintForPausedImageAnimationsIfNeeded(const IntRect& visibleRect);
     bool hasPausedImageAnimations() const { return m_hasPausedImageAnimations; }
     void setHasPausedImageAnimations(bool b) { m_hasPausedImageAnimations = b; }
@@ -328,6 +332,8 @@ private:
     unsigned m_renderBlockFlowHasMarkupTruncation : 1;
     unsigned m_renderBlockFlowLineLayoutPath : 2;
 
+    VisibleInViewportState m_visibleInViewportState { VisibilityUnknown };
+
     RenderObject* m_firstChild;
     RenderObject* m_lastChild;
 
index 67ded17..c810474 100644 (file)
@@ -77,7 +77,7 @@ protected:
     virtual void computeIntrinsicRatioInformation(FloatSize& intrinsicSize, double& intrinsicRatio) const override final;
     virtual bool foregroundIsKnownToBeOpaqueInRect(const LayoutRect& localRect, unsigned maxDepthToTest) const override;
 
-    virtual void styleDidChange(StyleDifference, const RenderStyle*) override final;
+    virtual void styleDidChange(StyleDifference, const RenderStyle*) override;
 
     virtual void imageChanged(WrappedImagePtr, const IntRect* = nullptr) override;
 
index d645fe8..423efcb 100644 (file)
@@ -63,6 +63,12 @@ void RenderMedia::layout()
         mediaElement().layoutSizeChanged();
 }
 
+void RenderMedia::styleDidChange(StyleDifference difference, const RenderStyle* oldStyle)
+{
+    RenderImage::styleDidChange(difference, oldStyle);
+    if (!oldStyle || style().visibility() != oldStyle->visibility())
+        mediaElement().visibilityDidChange();
+}
 
 } // namespace WebCore
 
index b64e325..c56559c 100644 (file)
@@ -57,6 +57,7 @@ private:
     virtual bool requiresForcedStyleRecalcPropagation() const override final { return true; }
 
     virtual bool shadowControlsNeedCustomLayoutMetrics() const override { return true; }
+    void styleDidChange(StyleDifference, const RenderStyle* oldStyle) override final;
 };
 
 } // namespace WebCore
index 8b74a66..7eee66d 100644 (file)
@@ -2196,6 +2196,18 @@ void RenderObject::setIsRenderFlowThread(bool isFlowThread)
         ensureRareData().setIsRenderFlowThread(isFlowThread);
 }
 
+void RenderObject::setIsRegisteredForVisibleInViewportCallback(bool registered)
+{
+    if (registered || hasRareData())
+        ensureRareData().setIsRegisteredForVisibleInViewportCallback(registered);
+}
+
+void RenderObject::setVisibleInViewportState(VisibleInViewportState visible)
+{
+    if (visible != VisibilityUnknown || hasRareData())
+        ensureRareData().setVisibleInViewportState(visible);
+}
+
 RenderObject::RareDataHash& RenderObject::rareDataMap()
 {
     static NeverDestroyed<RareDataHash> map;
index d81cebd..a011992 100644 (file)
@@ -504,6 +504,14 @@ public:
     bool isDragging() const { return m_bitfields.hasRareData() && rareData().isDragging(); }
     bool hasReflection() const { return m_bitfields.hasRareData() && rareData().hasReflection(); }
     bool isRenderFlowThread() const { return m_bitfields.hasRareData() && rareData().isRenderFlowThread(); }
+    bool isRegisteredForVisibleInViewportCallback() { return m_bitfields.hasRareData() && rareData().isRegisteredForVisibleInViewportCallback(); }
+
+    enum VisibleInViewportState {
+        VisibilityUnknown,
+        VisibleInViewport,
+        NotVisibleInViewport,
+    };
+    VisibleInViewportState visibleInViewportState() { return m_bitfields.hasRareData() ? rareData().visibleInViewportState() : VisibilityUnknown; }
 
     bool hasLayer() const { return m_bitfields.hasLayer(); }
 
@@ -613,6 +621,8 @@ public:
     void setIsDragging(bool);
     void setHasReflection(bool = true);
     void setIsRenderFlowThread(bool = true);
+    void setIsRegisteredForVisibleInViewportCallback(bool);
+    void setVisibleInViewportState(VisibleInViewportState);
 
     // Hook so that RenderTextControl can return the line height of its inner renderer.
     // For other renderers, the value is the same as lineHeight(false).
@@ -908,6 +918,13 @@ private:
         bool name() const { return m_##name; }\
         void set##Name(bool name) { m_##name = name; }\
 
+#define ADD_ENUM_BITFIELD(name, Name, Type, width) \
+    private:\
+        unsigned m_##name : width;\
+    public:\
+        Type name() const { return static_cast<Type>(m_##name); }\
+        void set##Name(Type name) { m_##name = static_cast<unsigned>(name); }\
+
     class RenderObjectBitfields {
         enum PositionedState {
             IsStaticallyPositioned = 0,
@@ -1012,12 +1029,18 @@ private:
             : m_isDragging(false)
             , m_hasReflection(false)
             , m_isRenderFlowThread(false)
+            , m_isRegisteredForVisibleInViewportCallback(false)
+            , m_visibleInViewportState(VisibilityUnknown)
         {
         }
-
         ADD_BOOLEAN_BITFIELD(isDragging, IsDragging);
         ADD_BOOLEAN_BITFIELD(hasReflection, HasReflection);
         ADD_BOOLEAN_BITFIELD(isRenderFlowThread, IsRenderFlowThread);
+
+        // From RenderElement
+        ADD_BOOLEAN_BITFIELD(isRegisteredForVisibleInViewportCallback, IsRegisteredForVisibleInViewportCallback);
+        ADD_ENUM_BITFIELD(visibleInViewportState, VisibleInViewportState, VisibleInViewportState, 2);
+
     };
     
     RenderObjectRareData rareData() const;
index 980aae7..74e9700 100644 (file)
@@ -1339,6 +1339,26 @@ ImageQualityController& RenderView::imageQualityController()
     return *m_imageQualityController;
 }
 
+void RenderView::registerForVisibleInViewportCallback(RenderElement& renderer)
+{
+    ASSERT(!m_visibleInViewportRenderers.contains(&renderer));
+    m_visibleInViewportRenderers.add(&renderer);
+}
+
+void RenderView::unregisterForVisibleInViewportCallback(RenderElement& renderer)
+{
+    ASSERT(m_visibleInViewportRenderers.contains(&renderer));
+    m_visibleInViewportRenderers.remove(&renderer);
+}
+
+void RenderView::updateVisibleViewportRect(const IntRect& visibleRect)
+{
+    resumePausedImageAnimationsIfNeeded(visibleRect);
+
+    for (auto* renderer : m_visibleInViewportRenderers)
+        renderer->visibleInViewportStateChanged(visibleRect.intersects(enclosingIntRect(renderer->absoluteClippedOverflowRect())) ? RenderElement::VisibleInViewport : RenderElement::NotVisibleInViewport);
+}
+
 void RenderView::addRendererWithPausedImageAnimations(RenderElement& renderer)
 {
     if (renderer.hasPausedImageAnimations()) {
index 40315f0..714b561 100644 (file)
@@ -225,6 +225,9 @@ public:
     void didCreateRenderer() { ++m_rendererCount; }
     void didDestroyRenderer() { --m_rendererCount; }
 
+    void updateVisibleViewportRect(const IntRect&);
+    void registerForVisibleInViewportCallback(RenderElement&);
+    void unregisterForVisibleInViewportCallback(RenderElement&);
     void resumePausedImageAnimationsIfNeeded(IntRect visibleRect);
     void addRendererWithPausedImageAnimations(RenderElement&);
     void removeRendererWithPausedImageAnimations(RenderElement&);
@@ -374,6 +377,7 @@ private:
     bool m_usesFirstLetterRules { false };
 
     HashSet<RenderElement*> m_renderersWithPausedImageAnimation;
+    HashSet<RenderElement*> m_visibleInViewportRenderers;
     Vector<RefPtr<RenderWidget>> m_protectedRenderWidgets;
 
 #if ENABLE(SERVICE_CONTROLS)
index 6c156e4..f557b0c 100644 (file)
@@ -2823,6 +2823,8 @@ void Internals::setMediaElementRestrictions(HTMLMediaElement* element, const Str
             restrictions |= MediaElementSession::MetadataPreloadingNotPermitted;
         if (equalIgnoringCase(restrictionString, "AutoPreloadingNotPermitted"))
             restrictions |= MediaElementSession::AutoPreloadingNotPermitted;
+        if (equalIgnoringCase(restrictionString, "InvisibleAutoplayNotPermitted"))
+            restrictions |= MediaElementSession::InvisibleAutoplayNotPermitted;
     }
     element->mediaSession().addBehaviorRestriction(restrictions);
 }