[GStreamer] Move GL video sink to its own GstBin sub-class
authorphiln@webkit.org <philn@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 28 Nov 2019 10:08:50 +0000 (10:08 +0000)
committerphiln@webkit.org <philn@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 28 Nov 2019 10:08:50 +0000 (10:08 +0000)
https://bugs.webkit.org/show_bug.cgi?id=204624

Reviewed by Xabier Rodriguez-Calvar.

Source/WebCore:

This reduces the MediaPlayerPrivate code-base and adds a good
separation of responsibility regarding GL video rendering. The
TextureCopier remains in the player because it's too specific.

* platform/GStreamer.cmake:
* platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp: Added.
(webkit_gl_video_sink_init):
(webKitGLVideoSinkFinalize):
(ensureGstGLContext):
(requestGLContext):
(webKitGLVideoSinkChangeState):
(webkit_gl_video_sink_class_init):
(webKitGLVideoSinkSetMediaPlayerPrivate):
(webKitGLVideoSinkProbePlatform):
* platform/graphics/gstreamer/GLVideoSinkGStreamer.h: Added.
* platform/graphics/gstreamer/GStreamerCommon.cpp:
(WebCore::initializeGStreamerAndRegisterWebKitElements):
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::load):
(WebCore::MediaPlayerPrivateGStreamer::changePipelineState):
(WebCore::MediaPlayerPrivateGStreamer::handleSyncMessage):
(WebCore::MediaPlayerPrivateGStreamer::createVideoSinkGL):
* platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:

Tools:

* Scripts/webkitpy/style/checker.py: White-list the new GLVideoSinkGStreamer GObject implementation.

git-svn-id: https://svn.webkit.org/repository/webkit/trunk@252917 268f45cc-cd09-0410-ab3c-d52691b4dbfc

12 files changed:
Source/WebCore/ChangeLog
Source/WebCore/platform/GStreamer.cmake
Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp [new file with mode: 0644]
Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.h [new file with mode: 0644]
Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.cpp
Source/WebCore/platform/graphics/gstreamer/GRefPtrGStreamer.h
Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp
Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.h
Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp
Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h
Tools/ChangeLog
Tools/Scripts/webkitpy/style/checker.py

index 08ae81c..3cfcaeb 100644 (file)
@@ -1,3 +1,34 @@
+2019-11-28  Philippe Normand  <pnormand@igalia.com>
+
+        [GStreamer] Move GL video sink to its own GstBin sub-class
+        https://bugs.webkit.org/show_bug.cgi?id=204624
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        This reduces the MediaPlayerPrivate code-base and adds a good
+        separation of responsibility regarding GL video rendering. The
+        TextureCopier remains in the player because it's too specific.
+
+        * platform/GStreamer.cmake:
+        * platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp: Added.
+        (webkit_gl_video_sink_init):
+        (webKitGLVideoSinkFinalize):
+        (ensureGstGLContext):
+        (requestGLContext):
+        (webKitGLVideoSinkChangeState):
+        (webkit_gl_video_sink_class_init):
+        (webKitGLVideoSinkSetMediaPlayerPrivate):
+        (webKitGLVideoSinkProbePlatform):
+        * platform/graphics/gstreamer/GLVideoSinkGStreamer.h: Added.
+        * platform/graphics/gstreamer/GStreamerCommon.cpp:
+        (WebCore::initializeGStreamerAndRegisterWebKitElements):
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
+        (WebCore::MediaPlayerPrivateGStreamer::load):
+        (WebCore::MediaPlayerPrivateGStreamer::changePipelineState):
+        (WebCore::MediaPlayerPrivateGStreamer::handleSyncMessage):
+        (WebCore::MediaPlayerPrivateGStreamer::createVideoSinkGL):
+        * platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h:
+
 2019-11-27  Zalan Bujtas  <zalan@apple.com>
 
         [LFC][IFC] Generate fewer InlineItems when new lines are not preserved
index 2c941f0..874d49a 100644 (file)
@@ -7,6 +7,7 @@ if (ENABLE_VIDEO OR ENABLE_WEB_AUDIO)
 
     list(APPEND WebCore_SOURCES
         platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp
+        platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp
         platform/graphics/gstreamer/GRefPtrGStreamer.cpp
         platform/graphics/gstreamer/GStreamerCommon.cpp
         platform/graphics/gstreamer/GstAllocatorFastMalloc.cpp
diff --git a/Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp b/Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp
new file mode 100644 (file)
index 0000000..8b00162
--- /dev/null
@@ -0,0 +1,352 @@
+/*
+ * Copyright (C) 2019 Igalia S.L
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Library General Public License for more details.
+ *
+ * You should have received a copy of the GNU Library General Public License
+ * aint with this library; see the file COPYING.LIB.  If not, write to
+ * the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+#include "config.h"
+#include "GLVideoSinkGStreamer.h"
+
+#if ENABLE(VIDEO) && USE(GSTREAMER_GL)
+
+#include "GLContext.h"
+#include "GStreamerCommon.h"
+#include "MediaPlayerPrivateGStreamer.h"
+
+#if USE(GLX)
+#include "GLContextGLX.h"
+#include <gst/gl/x11/gstgldisplay_x11.h>
+#endif
+
+#if USE(EGL)
+#include "GLContextEGL.h"
+#include <gst/gl/egl/gstgldisplay_egl.h>
+#endif
+
+#if PLATFORM(X11)
+#include "PlatformDisplayX11.h"
+#endif
+
+#if PLATFORM(WAYLAND)
+#include "PlatformDisplayWayland.h"
+#endif
+
+#if USE(WPE_RENDERER)
+#include "PlatformDisplayLibWPE.h"
+#endif
+
+// gstglapi.h may include eglplatform.h and it includes X.h, which
+// defines None, breaking MediaPlayer::None enum
+#if PLATFORM(X11) && GST_GL_HAVE_PLATFORM_EGL
+#undef None
+#endif // PLATFORM(X11) && GST_GL_HAVE_PLATFORM_EGL
+
+#include <gst/app/gstappsink.h>
+
+using namespace WebCore;
+
+struct _WebKitGLVideoSinkPrivate {
+    GRefPtr<GstElement> appSink;
+    GRefPtr<GstGLContext> glContext;
+    GRefPtr<GstGLDisplay> glDisplay;
+    GRefPtr<GstContext> glDisplayElementContext;
+    GRefPtr<GstContext> glAppElementContext;
+    MediaPlayerPrivateGStreamer* mediaPlayerPrivate;
+};
+
+GST_DEBUG_CATEGORY_STATIC(webkit_gl_video_sink_debug);
+#define GST_CAT_DEFAULT webkit_gl_video_sink_debug
+
+#define GST_GL_CAPS_FORMAT "{ RGBx, RGBA, I420, Y444, YV12, Y41B, Y42B, NV12, NV21, VUYA }"
+static GstStaticPadTemplate sinkTemplate = GST_STATIC_PAD_TEMPLATE("sink", GST_PAD_SINK, GST_PAD_ALWAYS, GST_STATIC_CAPS_ANY);
+
+#define WEBKIT_GL_VIDEO_SINK_GET_PRIVATE(obj) (G_TYPE_INSTANCE_GET_PRIVATE((obj), WEBKIT_TYPE_GL_VIDEO_SINK, WebKitGLVideoSinkPrivate))
+
+#define webkit_gl_video_sink_parent_class parent_class
+#define WEBKIT_GL_VIDEO_SINK_CATEGORY_INIT GST_DEBUG_CATEGORY_INIT(webkit_gl_video_sink_debug, "webkitglvideosink", 0, "GL video sink element");
+G_DEFINE_TYPE_WITH_CODE(WebKitGLVideoSink, webkit_gl_video_sink, GST_TYPE_BIN, WEBKIT_GL_VIDEO_SINK_CATEGORY_INIT);
+
+static void webkit_gl_video_sink_init(WebKitGLVideoSink* sink)
+{
+    sink->priv = WEBKIT_GL_VIDEO_SINK_GET_PRIVATE(sink);
+    new (sink->priv) WebKitGLVideoSinkPrivate();
+
+    sink->priv->appSink = gst_element_factory_make("appsink", "webkit-gl-video-appsink");
+    ASSERT(sink->priv->appSink);
+    g_object_set(sink->priv->appSink.get(), "enable-last-sample", FALSE, "emit-signals", TRUE, "max-buffers", 1, nullptr);
+
+    GstElement* upload = gst_element_factory_make("glupload", nullptr);
+    GstElement* colorconvert = gst_element_factory_make("glcolorconvert", nullptr);
+    ASSERT(upload);
+    ASSERT(colorconvert);
+    gst_bin_add_many(GST_BIN_CAST(sink), upload, colorconvert, sink->priv->appSink.get(), nullptr);
+
+    // Workaround until we can depend on GStreamer 1.16.2.
+    // https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/commit/8d32de090554cf29fe359f83aa46000ba658a693
+    // Forcing a color conversion to RGBA here allows glupload to internally use
+    // an uploader that adds a VideoMeta, through the TextureUploadMeta caps
+    // feature, without needing the patch above. However this specific caps
+    // feature is going to be removed from GStreamer so it is considered a
+    // short-term workaround. This code path most likely will have a negative
+    // performance impact on embedded platforms as well. Downstream embedders
+    // are highly encouraged to cherry-pick the patch linked above in their BSP
+    // and set the WEBKIT_GST_NO_RGBA_CONVERSION environment variable until
+    // GStreamer 1.16.2 is released.
+    // See also https://bugs.webkit.org/show_bug.cgi?id=201422
+    GRefPtr<GstCaps> caps;
+    if (webkitGstCheckVersion(1, 16, 2) || getenv("WEBKIT_GST_NO_RGBA_CONVERSION"))
+        caps = adoptGRef(gst_caps_from_string("video/x-raw, format = (string) " GST_GL_CAPS_FORMAT));
+    else {
+        GST_INFO_OBJECT(sink, "Forcing RGBA as GStreamer is not new enough.");
+        caps = adoptGRef(gst_caps_from_string("video/x-raw, format = (string) RGBA"));
+    }
+    gst_caps_set_features(caps.get(), 0, gst_caps_features_new(GST_CAPS_FEATURE_MEMORY_GL_MEMORY, nullptr));
+    g_object_set(sink->priv->appSink.get(), "caps", caps.get(), nullptr);
+
+    gst_element_link_many(upload, colorconvert, sink->priv->appSink.get(), nullptr);
+
+    GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(upload, "sink"));
+    gst_element_add_pad(GST_ELEMENT_CAST(sink), gst_ghost_pad_new("sink", pad.get()));
+}
+
+void webKitGLVideoSinkFinalize(GObject* object)
+{
+    ASSERT(WTF::isMainThread());
+
+    WebKitGLVideoSink* sink = WEBKIT_GL_VIDEO_SINK(object);
+    WebKitGLVideoSinkPrivate* priv = sink->priv;
+
+    if (priv->mediaPlayerPrivate)
+        g_signal_handlers_disconnect_by_data(priv->appSink.get(), priv->mediaPlayerPrivate);
+
+    // We used a placement new for construction, the destructor won't be called automatically.
+    priv->~_WebKitGLVideoSinkPrivate();
+
+    GST_CALL_PARENT(G_OBJECT_CLASS, finalize, (object));
+}
+
+static bool ensureGstGLContext(WebKitGLVideoSink* sink)
+{
+    WebKitGLVideoSinkPrivate* priv = sink->priv;
+
+    if (priv->glContext)
+        return true;
+
+    auto& sharedDisplay = PlatformDisplay::sharedDisplayForCompositing();
+
+    // The floating ref removal support was added in https://bugzilla.gnome.org/show_bug.cgi?id=743062.
+    bool shouldAdoptRef = webkitGstCheckVersion(1, 14, 0);
+    if (!priv->glDisplay) {
+#if PLATFORM(X11)
+#if USE(GLX)
+        if (is<PlatformDisplayX11>(sharedDisplay)) {
+            GST_DEBUG_OBJECT(sink, "Creating X11 shared GL display");
+            if (shouldAdoptRef)
+                priv->glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_x11_new_with_display(downcast<PlatformDisplayX11>(sharedDisplay).native())));
+            else
+                priv->glDisplay = GST_GL_DISPLAY(gst_gl_display_x11_new_with_display(downcast<PlatformDisplayX11>(sharedDisplay).native()));
+        }
+#elif USE(EGL)
+        if (is<PlatformDisplayX11>(sharedDisplay)) {
+            GST_DEBUG_OBJECT(sink, "Creating X11 shared EGL display");
+            if (shouldAdoptRef)
+                priv->glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayX11>(sharedDisplay).eglDisplay())));
+            else
+                priv->glDisplay = GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayX11>(sharedDisplay).eglDisplay()));
+        }
+#endif
+#endif
+
+#if PLATFORM(WAYLAND)
+        if (is<PlatformDisplayWayland>(sharedDisplay)) {
+            GST_DEBUG_OBJECT(sink, "Creating Wayland shared display");
+            if (shouldAdoptRef)
+                priv->glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayWayland>(sharedDisplay).eglDisplay())));
+            else
+                priv->glDisplay = GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayWayland>(sharedDisplay).eglDisplay()));
+        }
+#endif
+
+#if USE(WPE_RENDERER)
+        if (is<PlatformDisplayLibWPE>(sharedDisplay)) {
+            GST_DEBUG_OBJECT(sink, "Creating WPE shared EGL display");
+            if (shouldAdoptRef)
+                priv->glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayLibWPE>(sharedDisplay).eglDisplay())));
+            else
+                priv->glDisplay = GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayLibWPE>(sharedDisplay).eglDisplay()));
+        }
+#endif
+
+        ASSERT(priv->glDisplay);
+    }
+
+    GLContext* sharedContext = sharedDisplay.sharingGLContext();
+    // EGL and GLX are mutually exclusive, no need for ifdefs here.
+    GstGLPlatform glPlatform = sharedContext->isEGLContext() ? GST_GL_PLATFORM_EGL : GST_GL_PLATFORM_GLX;
+
+#if USE(OPENGL_ES)
+    GstGLAPI glAPI = GST_GL_API_GLES2;
+#elif USE(OPENGL)
+    GstGLAPI glAPI = GST_GL_API_OPENGL;
+#else
+    ASSERT_NOT_REACHED();
+#endif
+
+    PlatformGraphicsContext3D contextHandle = sharedContext->platformContext();
+    if (!contextHandle)
+        return false;
+
+    if (shouldAdoptRef)
+        priv->glContext = adoptGRef(gst_gl_context_new_wrapped(priv->glDisplay.get(), reinterpret_cast<guintptr>(contextHandle), glPlatform, glAPI));
+    else
+        priv->glContext = gst_gl_context_new_wrapped(priv->glDisplay.get(), reinterpret_cast<guintptr>(contextHandle), glPlatform, glAPI);
+
+    // Activate and fill the GStreamer wrapped context with the Webkit's shared one.
+    auto* previousActiveContext = GLContext::current();
+    sharedContext->makeContextCurrent();
+    if (gst_gl_context_activate(priv->glContext.get(), TRUE)) {
+        GUniqueOutPtr<GError> error;
+        if (!gst_gl_context_fill_info(priv->glContext.get(), &error.outPtr()))
+            GST_WARNING("Failed to fill in GStreamer context: %s", error->message);
+        gst_gl_context_activate(priv->glContext.get(), FALSE);
+    } else
+        GST_WARNING("Failed to activate GStreamer context %" GST_PTR_FORMAT, priv->glContext.get());
+    if (previousActiveContext)
+        previousActiveContext->makeContextCurrent();
+
+    return true;
+}
+
+GRefPtr<GstContext> requestGLContext(WebKitGLVideoSink* sink, const char* contextType)
+{
+    WebKitGLVideoSinkPrivate* priv = sink->priv;
+    if (!ensureGstGLContext(sink))
+        return nullptr;
+
+    if (!g_strcmp0(contextType, GST_GL_DISPLAY_CONTEXT_TYPE)) {
+        GstContext* displayContext = gst_context_new(GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);
+        gst_context_set_gl_display(displayContext, priv->glDisplay.get());
+        return adoptGRef(displayContext);
+    }
+
+    if (!g_strcmp0(contextType, "gst.gl.app_context")) {
+        GstContext* appContext = gst_context_new("gst.gl.app_context", TRUE);
+        GstStructure* structure = gst_context_writable_structure(appContext);
+#if GST_CHECK_VERSION(1, 12, 0)
+        gst_structure_set(structure, "context", GST_TYPE_GL_CONTEXT, priv->glContext.get(), nullptr);
+#else
+        gst_structure_set(structure, "context", GST_GL_TYPE_CONTEXT, priv->glContext.get(), nullptr);
+#endif
+        return adoptGRef(appContext);
+    }
+
+    return nullptr;
+}
+
+static GstStateChangeReturn webKitGLVideoSinkChangeState(GstElement* element, GstStateChange transition)
+{
+    WebKitGLVideoSink* sink = WEBKIT_GL_VIDEO_SINK(element);
+    WebKitGLVideoSinkPrivate* priv = sink->priv;
+
+#if GST_CHECK_VERSION(1, 14, 0)
+    GST_DEBUG_OBJECT(element, "%s", gst_state_change_get_name(transition));
+#endif
+
+    switch (transition) {
+    case GST_STATE_CHANGE_NULL_TO_READY:
+    case GST_STATE_CHANGE_READY_TO_READY:
+    case GST_STATE_CHANGE_READY_TO_PAUSED: {
+        if (!priv->glDisplayElementContext)
+            priv->glDisplayElementContext = requestGLContext(sink, GST_GL_DISPLAY_CONTEXT_TYPE);
+
+        if (priv->glDisplayElementContext)
+            gst_element_set_context(GST_ELEMENT_CAST(sink), priv->glDisplayElementContext.get());
+
+        if (!priv->glAppElementContext)
+            priv->glAppElementContext = requestGLContext(sink, "gst.gl.app_context");
+
+        if (priv->glAppElementContext)
+            gst_element_set_context(GST_ELEMENT_CAST(sink), priv->glAppElementContext.get());
+        break;
+    }
+    default:
+        break;
+    }
+
+    return GST_ELEMENT_CLASS(parent_class)->change_state(element, transition);
+}
+
+static void webkit_gl_video_sink_class_init(WebKitGLVideoSinkClass* klass)
+{
+    GObjectClass* objectClass = G_OBJECT_CLASS(klass);
+    GstElementClass* elementClass = GST_ELEMENT_CLASS(klass);
+
+    objectClass->finalize = webKitGLVideoSinkFinalize;
+
+    gst_element_class_add_pad_template(elementClass, gst_static_pad_template_get(&sinkTemplate));
+
+    gst_element_class_set_static_metadata(elementClass, "WebKit GL video sink", "Sink/Video", "Renders video", "Philippe Normand <philn@igalia.com>");
+
+    elementClass->change_state = GST_DEBUG_FUNCPTR(webKitGLVideoSinkChangeState);
+    g_type_class_add_private(klass, sizeof(WebKitGLVideoSinkPrivate));
+}
+
+void webKitGLVideoSinkSetMediaPlayerPrivate(WebKitGLVideoSink* sink, MediaPlayerPrivateGStreamer* player)
+{
+    WebKitGLVideoSinkPrivate* priv = sink->priv;
+
+    priv->mediaPlayerPrivate = player;
+    g_signal_connect(priv->appSink.get(), "new-sample", G_CALLBACK(+[](GstElement* sink, MediaPlayerPrivateGStreamer* player) -> GstFlowReturn {
+        GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink)));
+        player->triggerRepaint(sample.get());
+        return GST_FLOW_OK;
+    }), player);
+    g_signal_connect(priv->appSink.get(), "new-preroll", G_CALLBACK(+[](GstElement* sink, MediaPlayerPrivateGStreamer* player) -> GstFlowReturn {
+        GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_preroll(GST_APP_SINK(sink)));
+        player->triggerRepaint(sample.get());
+        return GST_FLOW_OK;
+    }), player);
+
+    GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(priv->appSink.get(), "sink"));
+    gst_pad_add_probe(pad.get(), static_cast<GstPadProbeType>(GST_PAD_PROBE_TYPE_PUSH | GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM | GST_PAD_PROBE_TYPE_EVENT_FLUSH), [] (GstPad*, GstPadProbeInfo* info,  gpointer userData) -> GstPadProbeReturn {
+        // In some platforms (e.g. OpenMAX on the Raspberry Pi) when a resolution change occurs the
+        // pipeline has to be drained before a frame with the new resolution can be decoded.
+        // In this context, it's important that we don't hold references to any previous frame
+        // (e.g. m_sample) so that decoding can continue.
+        // We are also not supposed to keep the original frame after a flush.
+        if (info->type & GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM) {
+            if (GST_QUERY_TYPE(GST_PAD_PROBE_INFO_QUERY(info)) != GST_QUERY_DRAIN)
+                return GST_PAD_PROBE_OK;
+            GST_DEBUG("Acting upon DRAIN query");
+        }
+        if (info->type & GST_PAD_PROBE_TYPE_EVENT_FLUSH) {
+            if (GST_EVENT_TYPE(GST_PAD_PROBE_INFO_EVENT(info)) != GST_EVENT_FLUSH_START)
+                return GST_PAD_PROBE_OK;
+            GST_DEBUG("Acting upon flush-start event");
+        }
+
+        auto* player = static_cast<MediaPlayerPrivateGStreamer*>(userData);
+        player->flushCurrentBuffer();
+        return GST_PAD_PROBE_OK;
+    }, player, nullptr);
+}
+
+bool webKitGLVideoSinkProbePlatform()
+{
+    return isGStreamerPluginAvailable("app") && isGStreamerPluginAvailable("opengl");
+}
+
+#endif
diff --git a/Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.h b/Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.h
new file mode 100644 (file)
index 0000000..456bcca
--- /dev/null
@@ -0,0 +1,58 @@
+/*
+ *  Copyright (C) 2019 Igalia, S.L
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Lesser General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Lesser General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Lesser General Public
+ *  License along with this library; if not, write to the Free Software
+ *  Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+ */
+
+#pragma once
+
+#if ENABLE(VIDEO) && USE(GSTREAMER_GL)
+
+#include <gst/gst.h>
+
+namespace WebCore {
+class MediaPlayerPrivateGStreamer;
+}
+
+G_BEGIN_DECLS
+
+#define WEBKIT_TYPE_GL_VIDEO_SINK            (webkit_gl_video_sink_get_type ())
+#define WEBKIT_GL_VIDEO_SINK(obj)            (G_TYPE_CHECK_INSTANCE_CAST ((obj), WEBKIT_TYPE_GL_VIDEO_SINK, WebKitGLVideoSink))
+#define WEBKIT_GL_VIDEO_SINK_CLASS(klass)    (G_TYPE_CHECK_CLASS_CAST ((klass), WEBKIT_TYPE_GL_VIDEO_SINK, WebKitGLVideoSinkClass))
+#define WEBKIT_IS_GL_VIDEO_SINK(obj)         (G_TYPE_CHECK_INSTANCE_TYPE ((obj), WEBKIT_TYPE_GL_VIDEO_SINK))
+#define WEBKIT_IS_GL_VIDEO_SINK_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), WEBKIT_TYPE_GL_VIDEO_SINK))
+
+typedef struct _WebKitGLVideoSink        WebKitGLVideoSink;
+typedef struct _WebKitGLVideoSinkClass   WebKitGLVideoSinkClass;
+typedef struct _WebKitGLVideoSinkPrivate WebKitGLVideoSinkPrivate;
+
+struct _WebKitGLVideoSink {
+    GstBin parent;
+
+    WebKitGLVideoSinkPrivate *priv;
+};
+
+struct _WebKitGLVideoSinkClass {
+    GstBinClass parentClass;
+};
+
+GType webkit_gl_video_sink_get_type(void);
+
+bool webKitGLVideoSinkProbePlatform();
+void webKitGLVideoSinkSetMediaPlayerPrivate(WebKitGLVideoSink*, WebCore::MediaPlayerPrivateGStreamer*);
+
+G_END_DECLS
+
+#endif // ENABLE(VIDEO) && USE(GSTREAMER_GL)
index b15f7f3..da50262 100644 (file)
@@ -45,6 +45,27 @@ template <> void derefGPtr<GstElement>(GstElement* ptr)
         gst_object_unref(ptr);
 }
 
+
+template <> GRefPtr<GstPlugin> adoptGRef(GstPlugin* ptr)
+{
+    ASSERT(!ptr || !g_object_is_floating(ptr));
+    return GRefPtr<GstPlugin>(ptr, GRefPtrAdopt);
+}
+
+template <> GstPlugin* refGPtr<GstPlugin>(GstPlugin* ptr)
+{
+    if (ptr)
+        gst_object_ref_sink(GST_OBJECT(ptr));
+
+    return ptr;
+}
+
+template <> void derefGPtr<GstPlugin>(GstPlugin* ptr)
+{
+    if (ptr)
+        gst_object_unref(ptr);
+}
+
 template <> GRefPtr<GstPad> adoptGRef(GstPad* ptr)
 {
     ASSERT(!ptr || !g_object_is_floating(ptr));
index a1965d4..1dc0529 100644 (file)
@@ -34,6 +34,10 @@ typedef struct _GstGLContext GstGLContext;
 
 namespace WTF {
 
+template<> GRefPtr<GstPlugin> adoptGRef(GstPlugin* ptr);
+template<> GstPlugin* refGPtr<GstPlugin>(GstPlugin* ptr);
+template<> void derefGPtr<GstPlugin>(GstPlugin* ptr);
+
 template<> GRefPtr<GstElement> adoptGRef(GstElement* ptr);
 template<> GstElement* refGPtr<GstElement>(GstElement* ptr);
 template<> void derefGPtr<GstElement>(GstElement* ptr);
index 2091776..089bf74 100644 (file)
@@ -23,6 +23,7 @@
 
 #if USE(GSTREAMER)
 
+#include "GLVideoSinkGStreamer.h"
 #include "GstAllocatorFastMalloc.h"
 #include "IntSize.h"
 #include "SharedBuffer.h"
@@ -296,6 +297,9 @@ bool initializeGStreamerAndRegisterWebKitElements()
 
 #if ENABLE(VIDEO)
         gst_element_register(0, "webkitwebsrc", GST_RANK_PRIMARY + 100, WEBKIT_TYPE_WEB_SRC);
+#if USE(GSTREAMER_GL)
+        gst_element_register(0, "webkitglvideosink", GST_RANK_PRIMARY, WEBKIT_TYPE_GL_VIDEO_SINK);
+#endif
 #endif
     });
     return true;
@@ -379,6 +383,14 @@ Ref<SharedBuffer> GstMappedBuffer::createSharedBuffer()
     return SharedBuffer::create(*this);
 }
 
+bool isGStreamerPluginAvailable(const char* name)
+{
+    GRefPtr<GstPlugin> plugin = adoptGRef(gst_registry_find_plugin(gst_registry_get(), name));
+    if (!plugin)
+        GST_WARNING("Plugin %s not found. Please check your GStreamer installation", name);
+    return plugin;
+}
+
 }
 
 #endif // USE(GSTREAMER)
index c813b78..969c7c3 100644 (file)
@@ -221,6 +221,8 @@ void disconnectSimpleBusMessageCallback(GstElement* pipeline);
 
 enum class GstVideoDecoderPlatform { ImxVPU, Video4Linux };
 
+bool isGStreamerPluginAvailable(const char* name);
+
 }
 
 #ifndef GST_BUFFER_DTS_OR_PTS
index e2249cd..579ed99 100644 (file)
 #endif // ENABLE(VIDEO_TRACK)
 
 #if USE(GSTREAMER_GL)
-#define TEXTURE_COPIER_COLOR_CONVERT_FLAG VideoTextureCopierGStreamer::ColorConversion::NoConvert
-#define GST_GL_CAPS_FORMAT "{ RGBx, RGBA, I420, Y444, YV12, Y41B, Y42B, NV12, NV21, VUYA }"
-
-#include <gst/app/gstappsink.h>
-
-#include "GLContext.h"
-#if USE(GLX)
-#include "GLContextGLX.h"
-#include <gst/gl/x11/gstgldisplay_x11.h>
-#endif
-
-#if USE(EGL)
-#include "GLContextEGL.h"
-#include <gst/gl/egl/gstgldisplay_egl.h>
-#endif
-
-#if PLATFORM(X11)
-#include "PlatformDisplayX11.h"
-#endif
-
-#if PLATFORM(WAYLAND)
-#include "PlatformDisplayWayland.h"
-#endif
-
-#if USE(WPE_RENDERER)
-#include "PlatformDisplayLibWPE.h"
-#endif
-
-// gstglapi.h may include eglplatform.h and it includes X.h, which
-// defines None, breaking MediaPlayer::None enum
-#if PLATFORM(X11) && GST_GL_HAVE_PLATFORM_EGL
-#undef None
-#endif // PLATFORM(X11) && GST_GL_HAVE_PLATFORM_EGL
+#include "GLVideoSinkGStreamer.h"
 #include "VideoTextureCopierGStreamer.h"
+
+#define TEXTURE_COPIER_COLOR_CONVERT_FLAG VideoTextureCopierGStreamer::ColorConversion::NoConvert
 #endif // USE(GSTREAMER_GL)
 
 #if USE(TEXTURE_MAPPER_GL)
@@ -459,15 +429,8 @@ MediaPlayerPrivateGStreamer::~MediaPlayerPrivateGStreamer()
 
     m_notifier->invalidate();
 
-    if (m_videoSink) {
+    if (m_videoSink)
         g_signal_handlers_disconnect_matched(m_videoSink.get(), G_SIGNAL_MATCH_DATA, 0, 0, nullptr, nullptr, this);
-#if USE(GSTREAMER_GL)
-        if (GST_IS_BIN(m_videoSink.get())) {
-            GRefPtr<GstElement> appsink = adoptGRef(gst_bin_get_by_name(GST_BIN_CAST(m_videoSink.get()), "webkit-gl-video-sink"));
-            g_signal_handlers_disconnect_by_data(appsink.get(), this);
-        }
-#endif
-    }
 
     if (m_volumeElement)
         g_signal_handlers_disconnect_matched(m_volumeElement.get(), G_SIGNAL_MATCH_DATA, 0, 0, nullptr, nullptr, this);
@@ -574,9 +537,6 @@ void MediaPlayerPrivateGStreamer::load(MediaStreamPrivate& stream)
     loadFull(String("mediastream://") + stream.id(), pipelineName);
     syncOnClock(false);
 
-#if USE(GSTREAMER_GL)
-    ensureGLVideoSinkContext();
-#endif
     m_player->play();
 }
 #endif
@@ -1182,11 +1142,6 @@ bool MediaPlayerPrivateGStreamer::changePipelineState(GstState newState)
     GST_DEBUG_OBJECT(pipeline(), "Changing state change to %s from %s with %s pending", gst_element_state_get_name(newState),
         gst_element_state_get_name(currentState), gst_element_state_get_name(pending));
 
-#if USE(GSTREAMER_GL)
-    if (currentState <= GST_STATE_READY && newState >= GST_STATE_PAUSED)
-        ensureGLVideoSinkContext();
-#endif
-
     GstStateChangeReturn setStateResult = gst_element_set_state(m_pipeline.get(), newState);
     GstState pausedOrPlaying = newState == GST_STATE_PLAYING ? GST_STATE_PAUSED : GST_STATE_PLAYING;
     if (currentState != pausedOrPlaying && setStateResult == GST_STATE_CHANGE_FAILURE)
@@ -1778,14 +1733,6 @@ bool MediaPlayerPrivateGStreamer::handleSyncMessage(GstMessage* message)
         return true;
     }
 
-#if USE(GSTREAMER_GL)
-    GRefPtr<GstContext> elementContext = adoptGRef(requestGLContext(contextType));
-    if (elementContext) {
-        gst_element_set_context(GST_ELEMENT(message->src), elementContext.get());
-        return true;
-    }
-#endif // USE(GSTREAMER_GL)
-
 #if ENABLE(ENCRYPTED_MEDIA)
     if (!g_strcmp0(contextType, "drm-preferred-decryption-system-id")) {
         if (isMainThread()) {
@@ -1829,125 +1776,10 @@ bool MediaPlayerPrivateGStreamer::handleSyncMessage(GstMessage* message)
         return true;
     }
 #endif // ENABLE(ENCRYPTED_MEDIA)
-    return false;
-}
-
-#if USE(GSTREAMER_GL)
-GstContext* MediaPlayerPrivateGStreamer::requestGLContext(const char* contextType)
-{
-    if (!ensureGstGLContext())
-        return nullptr;
-
-    if (!g_strcmp0(contextType, GST_GL_DISPLAY_CONTEXT_TYPE)) {
-        GstContext* displayContext = gst_context_new(GST_GL_DISPLAY_CONTEXT_TYPE, TRUE);
-        gst_context_set_gl_display(displayContext, gstGLDisplay());
-        return displayContext;
-    }
-
-    if (!g_strcmp0(contextType, "gst.gl.app_context")) {
-        GstContext* appContext = gst_context_new("gst.gl.app_context", TRUE);
-        GstStructure* structure = gst_context_writable_structure(appContext);
-#if GST_CHECK_VERSION(1, 12, 0)
-        gst_structure_set(structure, "context", GST_TYPE_GL_CONTEXT, gstGLContext(), nullptr);
-#else
-        gst_structure_set(structure, "context", GST_GL_TYPE_CONTEXT, gstGLContext(), nullptr);
-#endif
-        return appContext;
-    }
-
-    return nullptr;
-}
-
-bool MediaPlayerPrivateGStreamer::ensureGstGLContext()
-{
-    if (m_glContext)
-        return true;
-
-    auto& sharedDisplay = PlatformDisplay::sharedDisplayForCompositing();
-
-    // The floating ref removal support was added in https://bugzilla.gnome.org/show_bug.cgi?id=743062.
-    bool shouldAdoptRef = webkitGstCheckVersion(1, 14, 0);
-    if (!m_glDisplay) {
-#if PLATFORM(X11)
-#if USE(GLX)
-        if (is<PlatformDisplayX11>(sharedDisplay)) {
-            GST_DEBUG_OBJECT(pipeline(), "Creating X11 shared GL display");
-            if (shouldAdoptRef)
-                m_glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_x11_new_with_display(downcast<PlatformDisplayX11>(sharedDisplay).native())));
-            else
-                m_glDisplay = GST_GL_DISPLAY(gst_gl_display_x11_new_with_display(downcast<PlatformDisplayX11>(sharedDisplay).native()));
-        }
-#elif USE(EGL)
-        if (is<PlatformDisplayX11>(sharedDisplay)) {
-            GST_DEBUG_OBJECT(pipeline(), "Creating X11 shared EGL display");
-            if (shouldAdoptRef)
-                m_glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayX11>(sharedDisplay).eglDisplay())));
-            else
-                m_glDisplay = GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayX11>(sharedDisplay).eglDisplay()));
-        }
-#endif
-#endif
 
-#if PLATFORM(WAYLAND)
-        if (is<PlatformDisplayWayland>(sharedDisplay)) {
-            GST_DEBUG_OBJECT(pipeline(), "Creating Wayland shared display");
-            if (shouldAdoptRef)
-                m_glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayWayland>(sharedDisplay).eglDisplay())));
-            else
-                m_glDisplay = GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayWayland>(sharedDisplay).eglDisplay()));
-        }
-#endif
-
-#if USE(WPE_RENDERER)
-        if (is<PlatformDisplayLibWPE>(sharedDisplay)) {
-            GST_DEBUG_OBJECT(pipeline(), "Creating WPE shared EGL display");
-            if (shouldAdoptRef)
-                m_glDisplay = adoptGRef(GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayLibWPE>(sharedDisplay).eglDisplay())));
-            else
-                m_glDisplay = GST_GL_DISPLAY(gst_gl_display_egl_new_with_egl_display(downcast<PlatformDisplayLibWPE>(sharedDisplay).eglDisplay()));
-        }
-#endif
-
-        ASSERT(m_glDisplay);
-    }
-
-    GLContext* webkitContext = sharedDisplay.sharingGLContext();
-    // EGL and GLX are mutually exclusive, no need for ifdefs here.
-    GstGLPlatform glPlatform = webkitContext->isEGLContext() ? GST_GL_PLATFORM_EGL : GST_GL_PLATFORM_GLX;
-
-#if USE(OPENGL_ES)
-    GstGLAPI glAPI = GST_GL_API_GLES2;
-#elif USE(OPENGL)
-    GstGLAPI glAPI = GST_GL_API_OPENGL;
-#else
-    ASSERT_NOT_REACHED();
-#endif
-
-    PlatformGraphicsContext3D contextHandle = webkitContext->platformContext();
-    if (!contextHandle)
-        return false;
-
-    if (shouldAdoptRef)
-        m_glContext = adoptGRef(gst_gl_context_new_wrapped(m_glDisplay.get(), reinterpret_cast<guintptr>(contextHandle), glPlatform, glAPI));
-    else
-        m_glContext = gst_gl_context_new_wrapped(m_glDisplay.get(), reinterpret_cast<guintptr>(contextHandle), glPlatform, glAPI);
-
-    // Activate and fill the GStreamer wrapped context with the Webkit's shared one.
-    auto previousActiveContext = GLContext::current();
-    webkitContext->makeContextCurrent();
-    if (gst_gl_context_activate(m_glContext.get(), TRUE)) {
-        GUniqueOutPtr<GError> error;
-        if (!gst_gl_context_fill_info(m_glContext.get(), &error.outPtr()))
-            GST_WARNING("Failed to fill in GStreamer context: %s", error->message);
-        gst_gl_context_activate(m_glContext.get(), FALSE);
-    } else
-        GST_WARNING("Failed to activate GStreamer context %" GST_PTR_FORMAT, m_glContext.get());
-    if (previousActiveContext)
-        previousActiveContext->makeContextCurrent();
-
-    return true;
+    GST_DEBUG_OBJECT(pipeline(), "Unhandled %s need-context message for %s", contextType, GST_MESSAGE_SRC_NAME(message));
+    return false;
 }
-#endif // USE(GSTREAMER_GL)
 
 // Returns the size of the video
 FloatSize MediaPlayerPrivateGStreamer::naturalSize() const
@@ -3359,20 +3191,6 @@ void MediaPlayerPrivateGStreamer::repaintCancelledCallback(MediaPlayerPrivateGSt
 }
 
 #if USE(GSTREAMER_GL)
-GstFlowReturn MediaPlayerPrivateGStreamer::newSampleCallback(GstElement* sink, MediaPlayerPrivateGStreamer* player)
-{
-    GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink)));
-    player->triggerRepaint(sample.get());
-    return GST_FLOW_OK;
-}
-
-GstFlowReturn MediaPlayerPrivateGStreamer::newPrerollCallback(GstElement* sink, MediaPlayerPrivateGStreamer* player)
-{
-    GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_preroll(GST_APP_SINK(sink)));
-    player->triggerRepaint(sample.get());
-    return GST_FLOW_OK;
-}
-
 void MediaPlayerPrivateGStreamer::flushCurrentBuffer()
 {
     auto sampleLocker = holdLock(m_sampleMutex);
@@ -3595,118 +3413,17 @@ MediaPlayer::MovieLoadType MediaPlayerPrivateGStreamer::movieLoadType() const
 }
 
 #if USE(GSTREAMER_GL)
-GstElement* MediaPlayerPrivateGStreamer::createGLAppSink()
-{
-    GstElement* appsink = gst_element_factory_make("appsink", "webkit-gl-video-sink");
-    if (!appsink)
-        return nullptr;
-
-    g_object_set(appsink, "enable-last-sample", FALSE, "emit-signals", TRUE, "max-buffers", 1, nullptr);
-    g_signal_connect(appsink, "new-sample", G_CALLBACK(newSampleCallback), this);
-    g_signal_connect(appsink, "new-preroll", G_CALLBACK(newPrerollCallback), this);
-
-    GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(appsink, "sink"));
-    gst_pad_add_probe(pad.get(), static_cast<GstPadProbeType>(GST_PAD_PROBE_TYPE_PUSH | GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM | GST_PAD_PROBE_TYPE_EVENT_FLUSH), [] (GstPad*, GstPadProbeInfo* info,  gpointer userData) -> GstPadProbeReturn {
-        // In some platforms (e.g. OpenMAX on the Raspberry Pi) when a resolution change occurs the
-        // pipeline has to be drained before a frame with the new resolution can be decoded.
-        // In this context, it's important that we don't hold references to any previous frame
-        // (e.g. m_sample) so that decoding can continue.
-        // We are also not supposed to keep the original frame after a flush.
-        if (info->type & GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM) {
-            if (GST_QUERY_TYPE(GST_PAD_PROBE_INFO_QUERY(info)) != GST_QUERY_DRAIN)
-                return GST_PAD_PROBE_OK;
-            GST_DEBUG("Acting upon DRAIN query");
-        }
-        if (info->type & GST_PAD_PROBE_TYPE_EVENT_FLUSH) {
-            if (GST_EVENT_TYPE(GST_PAD_PROBE_INFO_EVENT(info)) != GST_EVENT_FLUSH_START)
-                return GST_PAD_PROBE_OK;
-            GST_DEBUG("Acting upon flush-start event");
-        }
-
-        auto* player = static_cast<MediaPlayerPrivateGStreamer*>(userData);
-        player->flushCurrentBuffer();
-        return GST_PAD_PROBE_OK;
-    }, this, nullptr);
-
-    return appsink;
-}
-
 GstElement* MediaPlayerPrivateGStreamer::createVideoSinkGL()
 {
-    gboolean result = TRUE;
-    GstElement* videoSink = gst_bin_new(nullptr);
-    GstElement* upload = gst_element_factory_make("glupload", nullptr);
-    GstElement* colorconvert = gst_element_factory_make("glcolorconvert", nullptr);
-    GstElement* appsink = createGLAppSink();
-
-    // glsinkbin is not used because it includes glcolorconvert which only process RGBA,
-    // but we can display YUV formats too.
-
-    if (!appsink || !upload || !colorconvert) {
-        GST_WARNING("Failed to create GstGL elements");
-        gst_object_unref(videoSink);
-
-        if (upload)
-            gst_object_unref(upload);
-        if (colorconvert)
-            gst_object_unref(colorconvert);
-        if (appsink)
-            gst_object_unref(appsink);
-
-        g_warning("WebKit wasn't able to find the GStreamer opengl plugin. Hardware-accelerated zero-copy video rendering can't be enabled without this plugin.");
+    if (!webKitGLVideoSinkProbePlatform()) {
+        g_warning("WebKit wasn't able to find the GL video sink dependencies. Hardware-accelerated zero-copy video rendering can't be enabled without this plugin.");
         return nullptr;
     }
 
-    gst_bin_add_many(GST_BIN(videoSink), upload, colorconvert, appsink, nullptr);
-
-    // Workaround until we can depend on GStreamer 1.16.2.
-    // https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/commit/8d32de090554cf29fe359f83aa46000ba658a693
-    // Forcing a color conversion to RGBA here allows glupload to internally use
-    // an uploader that adds a VideoMeta, through the TextureUploadMeta caps
-    // feature, without needing the patch above. However this specific caps
-    // feature is going to be removed from GStreamer so it is considered a
-    // short-term workaround. This code path most likely will have a negative
-    // performance impact on embedded platforms as well. Downstream embedders
-    // are highly encouraged to cherry-pick the patch linked above in their BSP
-    // and set the WEBKIT_GST_NO_RGBA_CONVERSION environment variable until
-    // GStreamer 1.16.2 is released.
-    // See also https://bugs.webkit.org/show_bug.cgi?id=201422
-    GRefPtr<GstCaps> caps;
-    if (webkitGstCheckVersion(1, 16, 2) || getenv("WEBKIT_GST_NO_RGBA_CONVERSION"))
-        caps = adoptGRef(gst_caps_from_string("video/x-raw, format = (string) " GST_GL_CAPS_FORMAT));
-    else {
-        GST_INFO_OBJECT(pipeline(), "Forcing RGBA as GStreamer is not new enough.");
-        caps = adoptGRef(gst_caps_from_string("video/x-raw, format = (string) RGBA"));
-    }
-    gst_caps_set_features(caps.get(), 0, gst_caps_features_new(GST_CAPS_FEATURE_MEMORY_GL_MEMORY, nullptr));
-    g_object_set(appsink, "caps", caps.get(), nullptr);
-
-    result &= gst_element_link_many(upload, colorconvert, appsink, nullptr);
-
-    GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(upload, "sink"));
-    gst_element_add_pad(videoSink, gst_ghost_pad_new("sink", pad.get()));
-
-    if (!result) {
-        GST_WARNING("Failed to link GstGL elements");
-        gst_object_unref(videoSink);
-        videoSink = nullptr;
-    }
-    return videoSink;
-}
-
-void MediaPlayerPrivateGStreamer::ensureGLVideoSinkContext()
-{
-    if (!m_glDisplayElementContext)
-        m_glDisplayElementContext = adoptGRef(requestGLContext(GST_GL_DISPLAY_CONTEXT_TYPE));
-
-    if (m_glDisplayElementContext)
-        gst_element_set_context(m_videoSink.get(), m_glDisplayElementContext.get());
-
-    if (!m_glAppElementContext)
-        m_glAppElementContext = adoptGRef(requestGLContext("gst.gl.app_context"));
-
-    if (m_glAppElementContext)
-        gst_element_set_context(m_videoSink.get(), m_glAppElementContext.get());
+    GstElement* sink = gst_element_factory_make("webkitglvideosink", nullptr);
+    ASSERT(sink);
+    webKitGLVideoSinkSetMediaPlayerPrivate(WEBKIT_GL_VIDEO_SINK(sink), this);
+    return sink;
 }
 #endif // USE(GSTREAMER_GL)
 
index 2adb0ff..2790ec3 100644 (file)
@@ -160,10 +160,6 @@ public:
     void setVolume(float) override;
     float volume() const override;
 
-#if USE(GSTREAMER_GL)
-    bool ensureGstGLContext();
-    GstContext* requestGLContext(const char* contextType);
-#endif
     void setMuted(bool) override;
     bool muted() const;
 
@@ -255,6 +251,11 @@ public:
     bool handleSyncMessage(GstMessage*);
     void handleMessage(GstMessage*);
 
+    void triggerRepaint(GstSample*);
+#if USE(GSTREAMER_GL)
+    void flushCurrentBuffer();
+#endif
+
 protected:
     enum MainThreadNotification {
         VideoChanged = 1 << 0,
@@ -285,14 +286,7 @@ protected:
 #endif
 
 #if USE(GSTREAMER_GL)
-    static GstFlowReturn newSampleCallback(GstElement*, MediaPlayerPrivateGStreamer*);
-    static GstFlowReturn newPrerollCallback(GstElement*, MediaPlayerPrivateGStreamer*);
-    void flushCurrentBuffer();
-    GstElement* createGLAppSink();
     GstElement* createVideoSinkGL();
-    GstGLContext* gstGLContext() const { return m_glContext.get(); }
-    GstGLDisplay* gstGLDisplay() const { return m_glDisplay.get(); }
-    void ensureGLVideoSinkContext();
 #endif
 
 #if USE(TEXTURE_MAPPER_GL)
@@ -311,7 +305,6 @@ protected:
 
     void setPipeline(GstElement*);
 
-    void triggerRepaint(GstSample*);
     void repaint();
     void cancelRepaint(bool destroying = false);
 
@@ -404,12 +397,7 @@ protected:
     bool m_destroying { false };
 
 #if USE(GSTREAMER_GL)
-    GRefPtr<GstGLContext> m_glContext;
-    GRefPtr<GstGLDisplay> m_glDisplay;
-    GRefPtr<GstContext> m_glDisplayElementContext;
-    GRefPtr<GstContext> m_glAppElementContext;
     std::unique_ptr<VideoTextureCopierGStreamer> m_videoTextureCopier;
-
     GRefPtr<GstGLColorConvert> m_colorConvert;
     GRefPtr<GstCaps> m_colorConvertInputCaps;
     GRefPtr<GstCaps> m_colorConvertOutputCaps;
index e2323ad..8b3f536 100644 (file)
@@ -1,3 +1,12 @@
+2019-11-28  Philippe Normand  <pnormand@igalia.com>
+
+        [GStreamer] Move GL video sink to its own GstBin sub-class
+        https://bugs.webkit.org/show_bug.cgi?id=204624
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        * Scripts/webkitpy/style/checker.py: White-list the new GLVideoSinkGStreamer GObject implementation.
+
 2019-11-27  Zalan Bujtas  <zalan@apple.com>
 
         [LFC] Unreviewed test gardening.
index 434a61e..13d1d48 100644 (file)
@@ -217,6 +217,8 @@ _PATH_RULES_SPECIFIER = [
     ([
       # These files define GObjects, which implies some definitions of
       # variables and functions containing underscores.
+      os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'GLVideoSinkGStreamer.cpp'),
+      os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'GLVideoSinkGStreamer.h'),
       os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'VideoSinkGStreamer.cpp'),
       os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'WebKitWebSourceGStreamer.cpp'),
       os.path.join('Source', 'WebCore', 'platform', 'audio', 'gstreamer', 'WebKitWebAudioSourceGStreamer.cpp'),