HTML5 audio .ended event not fired when app in background or phone screen is off
authorjer.noble@apple.com <jer.noble@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 20 Nov 2019 18:32:02 +0000 (18:32 +0000)
committerjer.noble@apple.com <jer.noble@apple.com@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Wed, 20 Nov 2019 18:32:02 +0000 (18:32 +0000)
https://bugs.webkit.org/show_bug.cgi?id=173332
<rdar://problem/32757402>

Reviewed by Eric Carlson.

Source/WebCore:

Test: media/audio-background-playback-playlist.html

In addition to the necessary WebKit part of this patch, there are behaviors which prevent
websites from enqueuing new playback while in the background. Namely, the platform will
prevent background playback from any application which is not currently the "Now Playing"
application, so in order to allow pages to switch sources, we must ensure we do not give
up "Now Playing" status. To do so, we will change the implementation of canProduceAudio()
to include any media element which previously could produce audio but currently has no
source.

Also, MediaElementSession::canShowControlsManager() will be modified to only check for
a RequireUserGestureToControlControlsManager restriction if the purpose passed in is
is "ControlsManager" and not "NowPlaying".

* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::canProduceAudio const):
* html/MediaElementSession.cpp:
(WebCore::MediaElementSession::canShowControlsManager const):

Source/WebKit:

When a WebPage goes from audible to inaudible, allow a short grace period before removing
the activity token, to give the page a chance to (e.g.) move to the next item in a playlist
before the process is suspended when in the background.

* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::close):
(WebKit::WebPageProxy::updateThrottleState):
(WebKit::WebPageProxy::clearAudibleActivity):
* UIProcess/WebPageProxy.h:

LayoutTests:

* media/audio-background-playback-playlist-expected.txt: Added.
* media/audio-background-playback-playlist.html: Added.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@252692 268f45cc-cd09-0410-ab3c-d52691b4dbfc

LayoutTests/ChangeLog
LayoutTests/media/audio-background-playback-playlist-expected.txt [new file with mode: 0644]
LayoutTests/media/audio-background-playback-playlist.html [new file with mode: 0644]
Source/WebCore/ChangeLog
Source/WebCore/html/HTMLMediaElement.cpp
Source/WebCore/html/MediaElementSession.cpp
Source/WebKit/ChangeLog
Source/WebKit/UIProcess/WebPageProxy.cpp
Source/WebKit/UIProcess/WebPageProxy.h

index 753e488..000aa67 100644 (file)
@@ -1,3 +1,14 @@
+2019-11-20  Jer Noble  <jer.noble@apple.com>
+
+        HTML5 audio .ended event not fired when app in background or phone screen is off
+        https://bugs.webkit.org/show_bug.cgi?id=173332
+        <rdar://problem/32757402>
+
+        Reviewed by Eric Carlson.
+
+        * media/audio-background-playback-playlist-expected.txt: Added.
+        * media/audio-background-playback-playlist.html: Added.
+
 2019-11-20  Simon Fraser  <simon.fraser@apple.com>
 
         REGRESSION (r252161): box-shadow with inset and rounded borders is clipped
diff --git a/LayoutTests/media/audio-background-playback-playlist-expected.txt b/LayoutTests/media/audio-background-playback-playlist-expected.txt
new file mode 100644 (file)
index 0000000..1d344b3
--- /dev/null
@@ -0,0 +1,21 @@
+
+RUN(internals.setMediaElementRestrictions(audio, "RequireUserGestureForAudioRateChange,RequireUserGestureToControlControlsManager"))
+RUN(audio.src = findMediaFile("audio", "content/test"))
+EVENT(canplaythrough)
+RUN(audio.play())
+EVENT(playing)
+EXPECTED (internals.bestMediaElementForShowingPlaybackControlsManager("NowPlaying") == '[object HTMLAudioElement]') OK
+RUN(internals.applicationDidEnterBackground(true))
+RUN(audio.currentTime = audio.duration - 0.1)
+EVENT(ended)
+RUN(audio.src = "")
+RUN(audio.load())
+EXPECTED (internals.bestMediaElementForShowingPlaybackControlsManager("NowPlaying") == '[object HTMLAudioElement]') OK
+RUN(audio.src = findMediaFile("audio", "content/test"))
+RUN(audio.load())
+EVENT(canplaythrough)
+RUN(audio.play())
+EVENT(playing)
+EXPECTED (internals.bestMediaElementForShowingPlaybackControlsManager("NowPlaying") == '[object HTMLAudioElement]') OK
+END OF TEST
+
diff --git a/LayoutTests/media/audio-background-playback-playlist.html b/LayoutTests/media/audio-background-playback-playlist.html
new file mode 100644 (file)
index 0000000..f729126
--- /dev/null
@@ -0,0 +1,35 @@
+<html>
+<head>
+    <script src=media-file.js></script>
+    <script src=video-test.js></script>
+    <script>
+    window.addEventListener('load', async event => {
+        window.audio = document.querySelector('audio');
+        run('internals.setMediaElementRestrictions(audio, "RequireUserGestureForAudioRateChange,RequireUserGestureToControlControlsManager")');
+        run('audio.src = findMediaFile("audio", "content/test")');
+
+        await waitFor(audio, 'canplaythrough');
+        runWithKeyDown('audio.play()');
+        await waitFor(audio, 'playing');
+        testExpected('internals.bestMediaElementForShowingPlaybackControlsManager("NowPlaying")', audio);
+        run('internals.applicationDidEnterBackground(true)');
+        run('audio.currentTime = audio.duration - 0.1')
+        await waitFor(audio, 'ended');
+        run('audio.src = ""');
+        run('audio.load()');
+        testExpected('internals.bestMediaElementForShowingPlaybackControlsManager("NowPlaying")', audio);
+        run('audio.src = findMediaFile("audio", "content/test")');
+        run('audio.load()');
+        await waitFor(audio, 'canplaythrough');
+        run('audio.play()');
+        await waitFor(audio, 'playing');
+        testExpected('internals.bestMediaElementForShowingPlaybackControlsManager("NowPlaying")', audio);
+        endTest();        
+    });
+    </script>
+</head>
+
+<body>
+    <audio controls><audio>
+</body>
+</html>
index 61e8978..866a40b 100644 (file)
@@ -1,3 +1,30 @@
+2019-11-20  Jer Noble  <jer.noble@apple.com>
+
+        HTML5 audio .ended event not fired when app in background or phone screen is off
+        https://bugs.webkit.org/show_bug.cgi?id=173332
+        <rdar://problem/32757402>
+
+        Reviewed by Eric Carlson.
+
+        Test: media/audio-background-playback-playlist.html
+
+        In addition to the necessary WebKit part of this patch, there are behaviors which prevent
+        websites from enqueuing new playback while in the background. Namely, the platform will
+        prevent background playback from any application which is not currently the "Now Playing"
+        application, so in order to allow pages to switch sources, we must ensure we do not give
+        up "Now Playing" status. To do so, we will change the implementation of canProduceAudio()
+        to include any media element which previously could produce audio but currently has no
+        source.
+
+        Also, MediaElementSession::canShowControlsManager() will be modified to only check for
+        a RequireUserGestureToControlControlsManager restriction if the purpose passed in is
+        is "ControlsManager" and not "NowPlaying".
+
+        * html/HTMLMediaElement.cpp:
+        (WebCore::HTMLMediaElement::canProduceAudio const):
+        * html/MediaElementSession.cpp:
+        (WebCore::MediaElementSession::canShowControlsManager const):
+
 2019-11-20  Simon Fraser  <simon.fraser@apple.com>
 
         REGRESSION (r252161): box-shadow with inset and rounded borders is clipped
index eddd287..f4e7d51 100644 (file)
@@ -7601,7 +7601,10 @@ bool HTMLMediaElement::canProduceAudio() const
     if (muted())
         return false;
 
-    return m_player && m_readyState >= HAVE_METADATA && hasAudio();
+    if (m_player && m_readyState >= HAVE_METADATA)
+        return hasAudio();
+
+    return hasEverHadAudio();
 }
 
 bool HTMLMediaElement::isSuspended() const
index 53f53bf..4c17268 100644 (file)
@@ -436,7 +436,7 @@ bool MediaElementSession::canShowControlsManager(PlaybackControlsPurpose purpose
         return true;
     }
 
-    if (client().presentationType() == Audio) {
+    if (client().presentationType() == Audio && purpose == PlaybackControlsPurpose::ControlsManager) {
         if (!hasBehaviorRestriction(RequireUserGestureToControlControlsManager) || m_element.document().processingUserGestureForMedia()) {
             INFO_LOG(LOGIDENTIFIER, "returning TRUE: audio element with user gesture");
             return true;
index 9f6efde..00e7a21 100644 (file)
@@ -1,3 +1,21 @@
+2019-11-20  Jer Noble  <jer.noble@apple.com>
+
+        HTML5 audio .ended event not fired when app in background or phone screen is off
+        https://bugs.webkit.org/show_bug.cgi?id=173332
+        <rdar://problem/32757402>
+
+        Reviewed by Eric Carlson.
+
+        When a WebPage goes from audible to inaudible, allow a short grace period before removing
+        the activity token, to give the page a chance to (e.g.) move to the next item in a playlist
+        before the process is suspended when in the background.
+
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::close):
+        (WebKit::WebPageProxy::updateThrottleState):
+        (WebKit::WebPageProxy::clearAudibleActivity):
+        * UIProcess/WebPageProxy.h:
+
 2019-11-19  Chris Dumez  <cdumez@apple.com>
 
         Unreviewed, fix webkitpy failures after r252655.
index 0d0648c..81b65c6 100644 (file)
@@ -269,6 +269,7 @@ static const unsigned wheelEventQueueSizeThreshold = 10;
 
 static const Seconds resetRecentCrashCountDelay = 30_s;
 static unsigned maximumWebProcessRelaunchAttempts = 1;
+static const Seconds audibleActivityClearDelay = 10_s;
 
 namespace WebKit {
 using namespace WebCore;
@@ -429,6 +430,7 @@ WebPageProxy::WebPageProxy(PageClient& pageClient, WebProcessProxy& process, Ref
     , m_notificationPermissionRequestManager(*this)
 #if PLATFORM(IOS_FAMILY)
     , m_alwaysRunsAtForegroundPriority(m_configuration->alwaysRunsAtForegroundPriority())
+    , m_audibleActivityTimer(RunLoop::main(), this, &WebPageProxy::clearAudibleActivity)
 #endif
     , m_initialCapitalizationEnabled(m_configuration->initialCapitalizationEnabled())
     , m_cpuLimit(m_configuration->cpuLimit())
@@ -1059,6 +1061,7 @@ void WebPageProxy::close()
     m_isAudibleActivity = nullptr;
     m_isCapturingActivity = nullptr;
     m_alwaysRunsAtForegroundPriorityActivity = nullptr;
+    m_audibleActivityTimer.stop();
 #endif
 
     stopAllURLSchemeTasks();
@@ -1949,9 +1952,11 @@ void WebPageProxy::updateThrottleState()
             RELEASE_LOG_IF_ALLOWED(ProcessSuspension, "updateThrottleState: UIProcess is taking a foreground assertion because we are playing audio");
             m_isAudibleActivity = m_process->throttler().foregroundActivity("View is playing audio"_s).moveToUniquePtr();
         }
+        m_audibleActivityTimer.stop();
     } else if (m_isAudibleActivity) {
-        RELEASE_LOG_IF_ALLOWED(ProcessSuspension, "updateThrottleState: UIProcess is releasing a foreground assertion because we are no longer playing audio");
-        m_isAudibleActivity = nullptr;
+        RELEASE_LOG_IF_ALLOWED(ProcessSuspension, "updateThrottleState: UIProcess will release a foreground assertion in %g seconds because we are no longer playing audio", audibleActivityClearDelay.seconds());
+        if (!m_audibleActivityTimer.isActive())
+            m_audibleActivityTimer.startOneShot(audibleActivityClearDelay);
     }
 
     bool isCapturingMedia = m_activityState.contains(ActivityState::IsCapturingMedia);
@@ -1977,6 +1982,14 @@ void WebPageProxy::updateThrottleState()
 #endif
 }
 
+#if PLATFORM(IOS_FAMILY)
+void WebPageProxy::clearAudibleActivity()
+{
+    RELEASE_LOG_IF_ALLOWED(ProcessSuspension, "updateThrottleState: UIProcess is releasing a foreground assertion because we are no longer playing audio");
+    m_isAudibleActivity = nullptr;
+}
+#endif
+
 void WebPageProxy::updateHiddenPageThrottlingAutoIncreases()
 {
     if (!m_preferences->hiddenPageDOMTimerThrottlingAutoIncreases())
index 66c9539..488ac80 100644 (file)
@@ -2169,6 +2169,7 @@ private:
 
 #if PLATFORM(IOS_FAMILY)
     static bool isInHardwareKeyboardMode();
+    void clearAudibleActivity();
 #endif
 
     void makeStorageSpaceRequest(WebCore::FrameIdentifier, const String& originIdentifier, const String& databaseName, const String& displayName, uint64_t currentQuota, uint64_t currentOriginUsage, uint64_t currentDatabaseUsage, uint64_t expectedUsage, CompletionHandler<void(uint64_t)>&&);
@@ -2300,6 +2301,7 @@ private:
     std::unique_ptr<ProcessThrottler::ForegroundActivity> m_isAudibleActivity;
     std::unique_ptr<ProcessThrottler::ForegroundActivity> m_isCapturingActivity;
     std::unique_ptr<ProcessThrottler::ForegroundActivity> m_alwaysRunsAtForegroundPriorityActivity;
+    RunLoop::Timer<WebPageProxy> m_audibleActivityTimer;
 #endif
     bool m_initialCapitalizationEnabled { false };
     Optional<double> m_cpuLimit;