[GStreamer][v4l2] Synchronous video texture flushing support

https://bugs.webkit.org/show_bug.cgi?id=195453

Reviewed by Xabier Rodriguez-Calvar.

The v4l2 video decoder currently requires that downstream users of
the graphics resources complete any pending draw call and release
resources before returning from the DRAIN query.

To accomplish this the player monitors the pipeline and whenever a
v4l2 decoder is added, synchronous video texture flushing support
is enabled. Additionally and for all decoder configurations, a
flush is performed before disposing of the player.

* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::playbinDeepElementAddedCallback):
Monitor elements added to the decodebin bin.
(WebCore::MediaPlayerPrivateGStreamer::decodebinElementAdded): Set
a flag if a v4l2 decoder was added in decodebin.
(WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Connect
to the deep-element-added signal so as to monitor pipeline
topology updates.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
(WebCore::MediaPlayerPrivateGStreamerBase::~MediaPlayerPrivateGStreamerBase):
Flush video texture before disposing of the player.
(WebCore::MediaPlayerPrivateGStreamerBase::flushCurrentBuffer):
Synchronously flush if the pipeline contains a v4l2 decoder.
(WebCore::MediaPlayerPrivateGStreamerBase::createGLAppSink): Monitor push events only.
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h:
* platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp:
(WebCore::TextureMapperPlatformLayerProxy::pushNextBuffer): New
boolean flag used mostly to trigger synchronous flush conditions.
(WebCore::TextureMapperPlatformLayerProxy::dropCurrentBufferWhilePreservingTexture):
Optionally drop the current buffer in a synchronous manner. By
default the method keeps operating asynchronously.
* platform/graphics/texmap/TextureMapperPlatformLayerProxy.h:

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@242701 268f45cc-cd09-0410-ab3c-d52691b4dbfc

diff --git a/Source/WebCore/ChangeLog b/Source/WebCore/ChangeLog
index 21ad29f..aacaf73 100644 (file)
--- a/Source/WebCore/ChangeLog
+++ b/Source/WebCore/ChangeLog
@@ -1,3 +1,43 @@
+2019-03-11  Philippe Normand  <pnormand@igalia.com>
+
+        [GStreamer][v4l2] Synchronous video texture flushing support
+        https://bugs.webkit.org/show_bug.cgi?id=195453
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        The v4l2 video decoder currently requires that downstream users of
+        the graphics resources complete any pending draw call and release
+        resources before returning from the DRAIN query.
+
+        To accomplish this the player monitors the pipeline and whenever a
+        v4l2 decoder is added, synchronous video texture flushing support
+        is enabled. Additionally and for all decoder configurations, a
+        flush is performed before disposing of the player.
+
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
+        (WebCore::MediaPlayerPrivateGStreamer::playbinDeepElementAddedCallback):
+        Monitor elements added to the decodebin bin.
+        (WebCore::MediaPlayerPrivateGStreamer::decodebinElementAdded): Set
+        a flag if a v4l2 decoder was added in decodebin.
+        (WebCore::MediaPlayerPrivateGStreamer::createGSTPlayBin): Connect
+        to the deep-element-added signal so as to monitor pipeline
+        topology updates.
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp:
+        (WebCore::MediaPlayerPrivateGStreamerBase::~MediaPlayerPrivateGStreamerBase):
+        Flush video texture before disposing of the player.
+        (WebCore::MediaPlayerPrivateGStreamerBase::flushCurrentBuffer):
+        Synchronously flush if the pipeline contains a v4l2 decoder.
+        (WebCore::MediaPlayerPrivateGStreamerBase::createGLAppSink): Monitor push events only.
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h:
+        * platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp:
+        (WebCore::TextureMapperPlatformLayerProxy::pushNextBuffer): New
+        boolean flag used mostly to trigger synchronous flush conditions.
+        (WebCore::TextureMapperPlatformLayerProxy::dropCurrentBufferWhilePreservingTexture):
+        Optionally drop the current buffer in a synchronous manner. By
+        default the method keeps operating asynchronously.
+        * platform/graphics/texmap/TextureMapperPlatformLayerProxy.h:
+
 2019-03-11  Antti Koivisto  <antti@apple.com>
 
         Rename contentOffsetInCompostingLayer to contentOffsetInCompositingLayer

diff --git a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
index f8f00f1..623663f 100644 (file)
--- a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
+++ b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
@@ -2445,6 +2445,15 @@ void MediaPlayerPrivateGStreamer::createGSTPlayBin(const gchar* playbinName, con
 
     g_object_set(m_pipeline.get(), "mute", m_player->muted(), nullptr);
 
+    g_signal_connect(GST_BIN_CAST(m_pipeline.get()), "deep-element-added", G_CALLBACK(+[](GstBin*, GstBin* subBin, GstElement* element, MediaPlayerPrivateGStreamer* player) {
+        GUniquePtr<char> binName(gst_element_get_name(GST_ELEMENT_CAST(subBin)));
+        if (!g_str_has_prefix(binName.get(), "decodebin"))
+            return;
+
+        GUniquePtr<char> elementName(gst_element_get_name(element));
+        player->m_isVideoDecoderVideo4Linux = g_str_has_prefix(elementName.get(), "v4l2");
+    }), this);
+
     g_signal_connect_swapped(m_pipeline.get(), "source-setup", G_CALLBACK(sourceSetupCallback), this);
     if (m_isLegacyPlaybin) {
         g_signal_connect_swapped(m_pipeline.get(), "video-changed", G_CALLBACK(videoChangedCallback), this);

diff --git a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp
index ad6200c..95a0bc0 100644 (file)
--- a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp
+++ b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp
@@ -269,6 +269,7 @@ MediaPlayerPrivateGStreamerBase::MediaPlayerPrivateGStreamerBase(MediaPlayer* pl
 
 MediaPlayerPrivateGStreamerBase::~MediaPlayerPrivateGStreamerBase()
 {
+    flushCurrentBuffer();
 #if USE(TEXTURE_MAPPER_GL) && USE(NICOSIA)
     downcast<Nicosia::ContentLayerTextureMapperImpl>(m_nicosiaLayer->impl()).invalidateClient();
 #endif
@@ -846,7 +847,6 @@ GstFlowReturn MediaPlayerPrivateGStreamerBase::newPrerollCallback(GstElement* si
 
 void MediaPlayerPrivateGStreamerBase::flushCurrentBuffer()
 {
-    GST_DEBUG_OBJECT(pipeline(), "Flushing video sample");
     auto sampleLocker = holdLock(m_sampleMutex);
 
     if (m_sample) {
@@ -858,12 +858,14 @@ void MediaPlayerPrivateGStreamerBase::flushCurrentBuffer()
     }
 
     auto proxyOperation =
-        [](TextureMapperPlatformLayerProxy& proxy)
+        [shouldWait = m_isVideoDecoderVideo4Linux, pipeline = pipeline()](TextureMapperPlatformLayerProxy& proxy)
         {
-            LockHolder locker(proxy.lock());
+            GST_DEBUG_OBJECT(pipeline, "Flushing video sample %s", shouldWait ? "synchronously" : "");
+            if (!shouldWait)
+                proxy.lock().lock();
 
             if (proxy.isActive())
-                proxy.dropCurrentBufferWhilePreservingTexture();
+                proxy.dropCurrentBufferWhilePreservingTexture(shouldWait);
         };
 
 #if USE(NICOSIA)
@@ -1010,7 +1012,7 @@ GstElement* MediaPlayerPrivateGStreamerBase::createGLAppSink()
     g_signal_connect(appsink, "new-preroll", G_CALLBACK(newPrerollCallback), this);
 
     GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(appsink, "sink"));
-    gst_pad_add_probe(pad.get(), static_cast<GstPadProbeType>(GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM | GST_PAD_PROBE_TYPE_EVENT_FLUSH), [] (GstPad*, GstPadProbeInfo* info,  gpointer userData) -> GstPadProbeReturn {
+    gst_pad_add_probe(pad.get(), static_cast<GstPadProbeType>(GST_PAD_PROBE_TYPE_PUSH | GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM | GST_PAD_PROBE_TYPE_EVENT_FLUSH), [] (GstPad*, GstPadProbeInfo* info,  gpointer userData) -> GstPadProbeReturn {
         // In some platforms (e.g. OpenMAX on the Raspberry Pi) when a resolution change occurs the
         // pipeline has to be drained before a frame with the new resolution can be decoded.
         // In this context, it's important that we don't hold references to any previous frame

diff --git a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h
index 20c28c4..686a18b 100644 (file)
--- a/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h
+++ b/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.h
@@ -282,6 +282,8 @@ protected:
     HashSet<uint32_t> m_handledProtectionEvents;
     bool m_waitingForKey { false };
 #endif
+
+    mutable bool m_isVideoDecoderVideo4Linux { false };
 };
 
 }

diff --git a/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp b/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp
index 5a7f126..c714f6d 100644 (file)
--- a/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp
+++ b/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.cpp
@@ -36,6 +36,7 @@
 #if USE(GLIB_EVENT_LOOP)
 #include <wtf/glib/RunLoopSourcePriority.h>
 #endif
+#include <wtf/Scope.h>
 
 static const Seconds releaseUnusedSecondsTolerance { 1_s };
 static const Seconds releaseUnusedBuffersTimerInterval = { 500_ms };
@@ -113,6 +114,7 @@ void TextureMapperPlatformLayerProxy::pushNextBuffer(std::unique_ptr<TextureMapp
 {
     ASSERT(m_lock.isHeld());
     m_pendingBuffer = WTFMove(newBuffer);
+    m_wasBufferDropped = false;
 
     if (m_compositor)
         m_compositor->onNewBufferAvailable();
@@ -187,9 +189,10 @@ void TextureMapperPlatformLayerProxy::swapBuffer()
         appendToUnusedBuffers(WTFMove(prevBuffer));
 }
 
-void TextureMapperPlatformLayerProxy::dropCurrentBufferWhilePreservingTexture()
+void TextureMapperPlatformLayerProxy::dropCurrentBufferWhilePreservingTexture(bool shouldWait)
 {
-    ASSERT(m_lock.isHeld());
+    if (!shouldWait)
+        ASSERT(m_lock.isHeld());
 
     if (m_pendingBuffer && m_pendingBuffer->hasManagedTexture()) {
         m_usedBuffers.append(WTFMove(m_pendingBuffer));
@@ -200,9 +203,17 @@ void TextureMapperPlatformLayerProxy::dropCurrentBufferWhilePreservingTexture()
         return;
 
     m_compositorThreadUpdateFunction =
-        [this] {
+        [this, shouldWait] {
             LockHolder locker(m_lock);
 
+            auto maybeNotifySynchronousOperation = WTF::makeScopeExit([this, shouldWait]() {
+                if (shouldWait) {
+                    LockHolder holder(m_wasBufferDroppedLock);
+                    m_wasBufferDropped = true;
+                    m_wasBufferDroppedCondition.notifyAll();
+                }
+            });
+
             if (!m_compositor || !m_targetLayer || !m_currentBuffer)
                 return;
 
@@ -214,7 +225,19 @@ void TextureMapperPlatformLayerProxy::dropCurrentBufferWhilePreservingTexture()
             if (prevBuffer->hasManagedTexture())
                 appendToUnusedBuffers(WTFMove(prevBuffer));
         };
+
+    if (shouldWait) {
+        LockHolder holder(m_wasBufferDroppedLock);
+        m_wasBufferDropped = false;
+    }
+
     m_compositorThreadUpdateTimer->startOneShot(0_s);
+    if (shouldWait) {
+        LockHolder holder(m_wasBufferDroppedLock);
+        m_wasBufferDroppedCondition.wait(m_wasBufferDroppedLock, [this] {
+            return m_wasBufferDropped;
+        });
+    }
 }
 
 bool TextureMapperPlatformLayerProxy::scheduleUpdateOnCompositorThread(Function<void()>&& updateFunction)

diff --git a/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.h b/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.h
index 683ec71..d38e6c2 100644 (file)
--- a/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.h
+++ b/Source/WebCore/platform/graphics/texmap/TextureMapperPlatformLayerProxy.h
@@ -28,6 +28,7 @@
 #if USE(COORDINATED_GRAPHICS)
 
 #include "TextureMapperGLHeaders.h"
+#include <wtf/Condition.h>
 #include <wtf/Function.h>
 #include <wtf/Lock.h>
 #include <wtf/RunLoop.h>
@@ -68,7 +69,7 @@ public:
     WEBCORE_EXPORT void invalidate();
 
     WEBCORE_EXPORT void swapBuffer();
-    void dropCurrentBufferWhilePreservingTexture();
+    void dropCurrentBufferWhilePreservingTexture(bool shouldWait = false);
 
     bool scheduleUpdateOnCompositorThread(Function<void()>&&);
 
@@ -85,6 +86,10 @@ private:
 
     Lock m_lock;
 
+    Lock m_wasBufferDroppedLock;
+    Condition m_wasBufferDroppedCondition;
+    bool m_wasBufferDropped { false };
+
     Vector<std::unique_ptr<TextureMapperPlatformLayerBuffer>> m_usedBuffers;
     std::unique_ptr<RunLoop::Timer<TextureMapperPlatformLayerProxy>> m_releaseUnusedBuffersTimer;