* [meta-fsl-arm][PATCH v2 0/3] Add patches for gstreamer1.0-plugins-bad 1.6
@ 2016-02-02 14:01 Yuqing Zhu
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance Yuqing Zhu
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Yuqing Zhu @ 2016-02-02 14:01 UTC (permalink / raw)
To: meta-freescale
--Add directviv to glimagesink to improve playback performance.
--Support video crop for glimagesink.
--Change wayland default res to 1024x768.
Yuqing Zhu (3):
gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve
playback performance
gstreamer1.0-plugins-bad: Support video crop for glimagesink
gstreamer1.0-plugins-bad: Change wayland default resolution to
1024x768
...imagesink-to-improve-playback-performance.patch | 454 +++++++++++++++++++++
...plugin-support-video-crop-for-glimagesink.patch | 155 +++++++
...in-Change-wayland-default-res-to-1024x768.patch | 41 ++
.../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 11 +-
4 files changed, 658 insertions(+), 3 deletions(-)
create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch
create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch
--
1.9.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance
2016-02-02 14:01 [meta-fsl-arm][PATCH v2 0/3] Add patches for gstreamer1.0-plugins-bad 1.6 Yuqing Zhu
@ 2016-02-02 14:01 ` Yuqing Zhu
2016-03-29 7:32 ` Carlos Rafael Giani
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 2/3] gstreamer1.0-plugins-bad: Support video crop for glimagesink Yuqing Zhu
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 3/3] gstreamer1.0-plugins-bad: Change wayland default resolution to 1024x768 Yuqing Zhu
2 siblings, 1 reply; 8+ messages in thread
From: Yuqing Zhu @ 2016-02-02 14:01 UTC (permalink / raw)
To: meta-freescale
1. Add a physical buffer uploader in glupload plugin and using viv direct
texture to bind physical continious buffer with texture to avoid memory
copy from videobuffer to texture to gain good performance.
2. Reduce glimagesink load latency by override glimagesink ALLOCATION query to
avoid memory copy.
Signed-off-by: Yuqing Zhu <b54851@freescale.com>
---
...imagesink-to-improve-playback-performance.patch | 454 +++++++++++++++++++++
.../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 9 +-
2 files changed, 460 insertions(+), 3 deletions(-)
create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
new file mode 100755
index 0000000..2739355
--- /dev/null
+++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
@@ -0,0 +1,454 @@
+From 587db892fdaac18780c8024e5a976ea182f36ff4 Mon Sep 17 00:00:00 2001
+From: Haihua Hu <b55597@freescale.com>
+Date: Tue, 13 Oct 2015 09:33:54 +0800
+Subject: [PATCH 1/2] [glplugin] Add directviv to glimagesink to improve
+ playback performance
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+1.Add a physical buffer uploader in glupload plugin and using viv direct
+ texture to bind physical continious buffer with texture to avoid memory
+ copy from videobuffer to texture to gain good performance.
+2.Reduce glimagesink load latency by override glimagesink ALLOCATION query to
+ avoid memory copy.
+
+Upstream-Status: Inappropriate [i.MX specific]
+
+Signed-off-by: Haihua Hu <b55597@freescale.com>
+---
+ ext/gl/gstglimagesink.c | 5 ++
+ ext/gl/gstgluploadelement.c | 14 +++
+ gst-libs/gst/gl/Makefile.am | 2 +
+ gst-libs/gst/gl/gstglupload.c | 118 ++++++++++++++++++++++++-
+ gst-libs/gst/gl/gstglvivdirecttexture.c | 143 +++++++++++++++++++++++++++++++
+ gst-libs/gst/gl/gstglvivdirecttexture.h | 35 ++++++++
+ 6 files changed, 314 insertions(+), 3 deletions(-)
+ create mode 100644 gst-libs/gst/gl/gstglvivdirecttexture.c
+ create mode 100644 gst-libs/gst/gl/gstglvivdirecttexture.h
+
+diff --git a/ext/gl/gstglimagesink.c b/ext/gl/gstglimagesink.c
+index f629df7..1e5dc05 100644
+--- a/ext/gl/gstglimagesink.c
++++ b/ext/gl/gstglimagesink.c
+@@ -896,6 +896,11 @@ gst_glimage_sink_query (GstBaseSink * bsink, GstQuery * query)
+ res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
+ break;
+ }
++ case GST_QUERY_ALLOCATION:
++ {
++ res = gst_glimage_sink_propose_allocation(bsink, query);
++ break;
++ }
+ default:
+ res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
+ break;
+diff --git a/ext/gl/gstgluploadelement.c b/ext/gl/gstgluploadelement.c
+index 83ff24c..e74da3a 100644
+--- a/ext/gl/gstgluploadelement.c
++++ b/ext/gl/gstgluploadelement.c
+@@ -208,6 +208,8 @@ gst_gl_upload_element_prepare_output_buffer (GstBaseTransform * bt,
+ GstGLUploadElement *upload = GST_GL_UPLOAD_ELEMENT (bt);
+ GstGLUploadReturn ret;
+ GstBaseTransformClass *bclass;
++ GstVideoInfo info;
++ GstCaps *caps;
+
+ bclass = GST_BASE_TRANSFORM_GET_CLASS (bt);
+
+@@ -231,6 +233,18 @@ gst_gl_upload_element_prepare_output_buffer (GstBaseTransform * bt,
+ return GST_FLOW_ERROR;
+ }
+
++ /* check if the output buffer's meta format equals to the caps format */
++ GstVideoMeta *vmeta = gst_buffer_get_video_meta(*outbuf);
++ gst_video_info_from_caps (&info, upload->out_caps);
++ if(vmeta->format != GST_VIDEO_INFO_FORMAT (&info))
++ {
++ caps = gst_caps_copy(upload->out_caps);
++ gst_caps_set_simple (caps, "format", G_TYPE_STRING,
++ gst_video_format_to_string (vmeta->format), NULL);
++ gst_caps_replace (&upload->out_caps, caps);
++ gst_base_transform_update_src_caps (upload, caps);
++ }
++
+ /* basetransform doesn't unref if they're the same */
+ if (buffer == *outbuf)
+ gst_buffer_unref (*outbuf);
+diff --git a/gst-libs/gst/gl/Makefile.am b/gst-libs/gst/gl/Makefile.am
+index 36f3bf7..402e37a 100644
+--- a/gst-libs/gst/gl/Makefile.am
++++ b/gst-libs/gst/gl/Makefile.am
+@@ -30,6 +30,7 @@ libgstgl_@GST_API_VERSION@_la_SOURCES = \
+ gstglsyncmeta.c \
+ gstglviewconvert.c \
+ gstgloverlaycompositor.c \
++ gstglvivdirecttexture.c \
+ utils/opengl_versions.h
+
+ libgstgl_@GST_API_VERSION@includedir = $(includedir)/gstreamer-@GST_API_VERSION@/gst/gl
+@@ -56,6 +57,7 @@ libgstgl_@GST_API_VERSION@include_HEADERS = \
+ gstglviewconvert.h \
+ gstgloverlaycompositor.h \
+ gstgl_fwd.h \
++ gstglvivdirecttexture.h \
+ gl.h
+
+ libgstgl_@GST_API_VERSION@_la_LIBADD = \
+diff --git a/gst-libs/gst/gl/gstglupload.c b/gst-libs/gst/gl/gstglupload.c
+index 0addca1..acaa329 100644
+--- a/gst-libs/gst/gl/gstglupload.c
++++ b/gst-libs/gst/gl/gstglupload.c
+@@ -23,6 +23,7 @@
+ #endif
+
+ #include <stdio.h>
++#include <gst/gl/gstglvivdirecttexture.h>
+
+ #include "gl.h"
+ #include "gstglupload.h"
+@@ -47,7 +48,7 @@
+ #define USING_GLES2(context) (gst_gl_context_check_gl_version (context, GST_GL_API_GLES2, 2, 0))
+ #define USING_GLES3(context) (gst_gl_context_check_gl_version (context, GST_GL_API_GLES2, 3, 0))
+
+-GST_DEBUG_CATEGORY_STATIC (gst_gl_upload_debug);
++GST_DEBUG_CATEGORY (gst_gl_upload_debug);
+ #define GST_CAT_DEFAULT gst_gl_upload_debug
+
+ #define DEBUG_INIT \
+@@ -698,6 +699,114 @@ static const UploadMethod _upload_meta_upload = {
+ &_upload_meta_upload_free
+ };
+
++struct PhyBufferUpload
++{
++ GstGLUpload *upload;
++};
++
++static gpointer
++_physical_buffer_upload_new(GstGLUpload *upload)
++{
++ struct PhyBufferUpload *phybuffer = g_new0 (struct PhyBufferUpload, 1);
++
++ phybuffer->upload = upload;
++
++ return phybuffer;
++}
++
++static GstCaps *
++_physical_buffer_upload_transform_caps(GstGLContext *context,
++ GstPadDirection direction, GstCaps *caps)
++{
++ GstCaps *ret;
++
++ if (direction == GST_PAD_SINK) {
++ ret = _set_caps_features (caps, GST_CAPS_FEATURE_MEMORY_GL_MEMORY);
++ gst_caps_set_simple (ret, "format", G_TYPE_STRING, "RGBA", NULL);
++ } else {
++ ret = gst_caps_from_string (GST_VIDEO_CAPS_MAKE_WITH_FEATURES
++ (GST_CAPS_FEATURE_MEMORY_SYSTEM_MEMORY,GST_GL_DIRECTVIV_FORMAT));
++ }
++
++ return ret;
++}
++
++static gboolean
++_physical_buffer_upload_accept(gpointer impl, GstBuffer *buffer,
++ GstCaps *in_caps, GstCaps *out_caps)
++{
++ GstCapsFeatures *features;
++
++ features = gst_caps_get_features (out_caps, 0);
++ if (!gst_caps_features_contains (features, GST_CAPS_FEATURE_MEMORY_GL_MEMORY))
++ return FALSE;
++
++ return gst_is_physical_buffer(buffer);
++}
++
++static void
++_physical_buffer_upload_propose_allocation(gpointer impl, GstQuery *decide_query,
++ GstQuery *query)
++{
++ gst_query_add_allocation_meta (query, GST_VIDEO_META_API_TYPE, 0);
++}
++
++static GstGLUploadReturn
++_physical_buffer_upload_perform(gpointer impl, GstBuffer *buffer, GstBuffer **outbuf)
++{
++ struct PhyBufferUpload *phyBuffer = impl;
++ GstVideoInfo *info;
++ gint n_mem;
++
++ info = &phyBuffer->upload->priv->out_info;
++ n_mem = GST_VIDEO_INFO_N_PLANES (info);
++ GST_LOG_OBJECT (phyBuffer->upload, "Attempting viv direct upload");
++ *outbuf = gst_buffer_new ();
++ gst_gl_memory_setup_buffer (phyBuffer->upload->context,
++ NULL, info, NULL, *outbuf);
++
++ GstGLMemory *out_gl_mem =
++ (GstGLMemory *) gst_buffer_peek_memory (*outbuf, 0);
++
++ gst_gl_viv_direct_bind_gstbuffer(phyBuffer->upload->context, out_gl_mem->tex_id,
++ &phyBuffer->upload->priv->in_info, buffer);
++
++ gst_buffer_add_video_meta_full (*outbuf, 0,
++ GST_VIDEO_INFO_FORMAT (info), GST_VIDEO_INFO_WIDTH (info),
++ GST_VIDEO_INFO_HEIGHT (info), n_mem, info->offset, info->stride);
++
++ return GST_GL_UPLOAD_DONE;
++}
++
++static void
++_physical_buffer_upload_release(gpointer impl, GstBuffer *buffer)
++{
++}
++
++static void
++_physical_buffer_upload_free(gpointer impl)
++{
++ struct PhyBufferUpload *phyBuffer = impl;
++
++ g_free(phyBuffer);
++}
++
++static GstStaticCaps _physical_buffer_upload_caps =
++GST_STATIC_CAPS (GST_VIDEO_CAPS_MAKE (GST_GL_DIRECTVIV_FORMAT));
++
++static const UploadMethod _physical_buffer_upload = {
++ "Physical buffer",
++ 0,
++ &_physical_buffer_upload_caps,
++ &_physical_buffer_upload_new,
++ &_physical_buffer_upload_transform_caps,
++ &_physical_buffer_upload_accept,
++ &_physical_buffer_upload_propose_allocation,
++ &_physical_buffer_upload_perform,
++ &_physical_buffer_upload_release,
++ &_physical_buffer_upload_free
++};
++
+ struct RawUploadFrame
+ {
+ gint ref_count;
+@@ -813,6 +922,7 @@ _raw_data_upload_perform (gpointer impl, GstBuffer * buffer,
+ struct RawUpload *raw = impl;
+ int i;
+ GstVideoInfo *in_info = &raw->upload->priv->in_info;
++
+ guint max_planes = GST_VIDEO_INFO_N_PLANES (in_info);
+
+ /* Support stereo views for separated multiview mode */
+@@ -870,7 +980,9 @@ static const UploadMethod *upload_methods[] = { &_gl_memory_upload,
+ #if GST_GL_HAVE_PLATFORM_EGL
+ &_egl_image_upload,
+ #endif
+- &_upload_meta_upload, &_raw_data_upload
++ &_upload_meta_upload,
++ &_physical_buffer_upload,
++ &_raw_data_upload
+ };
+
+ static GMutex upload_global_lock;
+@@ -998,7 +1110,7 @@ gst_gl_upload_transform_caps (GstGLContext * context, GstPadDirection direction,
+
+
+ if (filter) {
+- result = gst_caps_intersect_full (filter, tmp, GST_CAPS_INTERSECT_FIRST);
++ result = gst_caps_intersect_full (tmp, filter, GST_CAPS_INTERSECT_FIRST);
+ gst_caps_unref (tmp);
+ } else {
+ result = tmp;
+diff --git a/gst-libs/gst/gl/gstglvivdirecttexture.c b/gst-libs/gst/gl/gstglvivdirecttexture.c
+new file mode 100644
+index 0000000..c19b617
+--- /dev/null
++++ b/gst-libs/gst/gl/gstglvivdirecttexture.c
+@@ -0,0 +1,143 @@
++/*
++ * GStreamer
++ * Copyright (c) 2015, Freescale Semiconductor, Inc.
++ *
++ * This library is free software; you can redistribute it and/or
++ * modify it under the terms of the GNU Library General Public
++ * License as published by the Free Software Foundation; either
++ * version 2 of the License, or (at your option) any later version.
++ *
++ * This library is distributed in the hope that it will be useful,
++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
++ * Library General Public License for more details.
++ *
++ * You should have received a copy of the GNU Library General Public
++ * License along with this library; if not, write to the
++ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
++ * Boston, MA 02110-1301, USA.
++ */
++
++#ifdef HAVE_CONFIG_H
++#include "config.h"
++#endif
++
++#include "gl.h"
++
++GST_DEBUG_CATEGORY_EXTERN (gst_gl_upload_debug);
++#define GST_CAT_DEFAULT gst_gl_upload_debug
++
++typedef struct {
++ guint tex_id;
++ guint w;
++ guint h;
++ guint fmt;
++ void *vaddr;
++ guint paddr;
++ gboolean ret;
++} GstVivDirectTexture;
++
++gboolean
++gst_is_physical_buffer (GstBuffer *buffer)
++{
++
++ GstMemory *mem;
++
++ mem = gst_buffer_peek_memory (buffer, 0);
++ if (!mem->allocator)
++ return FALSE;
++
++ return g_type_check_instance_is_a (mem->allocator, g_type_from_name("GstAllocatorPhyMem"));
++}
++
++static void
++_do_viv_direct_tex_bind_mem (GstGLContext * context, GstVivDirectTexture * viv_tex)
++{
++ GST_DEBUG ("viv direct upload, tex_id %d, fmt: %d, res: (%dx%d)", viv_tex->tex_id, viv_tex->fmt, viv_tex->w, viv_tex->h);
++ GST_DEBUG ("Physical memory buffer, vaddr: %p, paddr: %p", viv_tex->vaddr, viv_tex->paddr);
++
++ glBindTexture (GL_TEXTURE_2D, viv_tex->tex_id);
++ glTexDirectVIVMap (GL_TEXTURE_2D, viv_tex->w, viv_tex->h, viv_tex->fmt, &viv_tex->vaddr, &viv_tex->paddr);
++ glTexDirectInvalidateVIV (GL_TEXTURE_2D);
++ viv_tex->ret = TRUE;
++
++ return;
++}
++
++gboolean
++gst_gl_viv_direct_bind_gstbuffer (GstGLContext * context, guint tex_id, GstVideoInfo * info, GstBuffer * buffer)
++{
++ typedef struct {
++ guint8 *vaddr;
++ guint8 *paddr;
++ guint8 *caddr;
++ gsize size;
++ gpointer *user_data;
++ } PhyMemBlock;
++ //Note: structure PhyMemBlock is copied from gst1.0-fsl-plugin/libs/allocator/gstallocatorphymem.h
++
++ typedef struct {
++ GstMemory mem;
++ guint8 *vaddr;
++ guint8 *paddr;
++ PhyMemBlock block;
++ } GstMemoryPhy;
++ //Note: structure GstMemoryPhy is copied from gst1.0-fsl-plugin/libs/allocator/gstallocatorphymem.c
++
++ GstMemory *mem = gst_buffer_peek_memory (buffer, 0);
++ GstMemoryPhy *memphy = (GstMemoryPhy*) mem;
++ PhyMemBlock *memblk = &memphy->block;
++
++ GstVideoFormat fmt = GST_VIDEO_INFO_FORMAT (info);
++ gint width, height;
++ GstVideoMeta *vmeta = gst_buffer_get_video_meta (buffer);
++ if (vmeta && (fmt == GST_VIDEO_FORMAT_I420 || fmt == GST_VIDEO_FORMAT_NV12)) {
++ width = vmeta->stride[0];
++ height = vmeta->offset[1] / width;
++ }
++ else {
++ width = GST_VIDEO_INFO_WIDTH (info);
++ height = GST_VIDEO_INFO_HEIGHT (info);
++ }
++
++ guint viv_fmt;
++ switch (fmt) {
++ case GST_VIDEO_FORMAT_I420:
++ viv_fmt = GL_VIV_I420;
++ break;
++ case GST_VIDEO_FORMAT_YV12:
++ viv_fmt = GL_VIV_YV12;
++ break;
++ case GST_VIDEO_FORMAT_NV12:
++ viv_fmt = GL_VIV_NV12;
++ break;
++ case GST_VIDEO_FORMAT_NV21:
++ viv_fmt = GL_VIV_NV21;
++ break;
++ case GST_VIDEO_FORMAT_YUY2:
++ viv_fmt = GL_VIV_YUY2;
++ break;
++ case GST_VIDEO_FORMAT_UYVY:
++ viv_fmt = GL_VIV_UYVY;
++ break;
++ case GST_VIDEO_FORMAT_RGBA:
++ viv_fmt = GL_RGBA;
++ break;
++ case GST_VIDEO_FORMAT_BGRA:
++ viv_fmt = GL_BGRA_EXT;
++ break;
++ case GST_VIDEO_FORMAT_RGB16:
++ viv_fmt = GL_RGB565_OES;
++ break;
++ default:
++ GST_ERROR ("Not supported format %d for viv direct texture upload.", fmt);
++ viv_fmt = GL_NONE;
++ return FALSE;
++ }
++
++ GstVivDirectTexture viv_tex = {tex_id, width, height, viv_fmt, memblk->vaddr, memblk->paddr, FALSE};
++ gst_gl_context_thread_add (context, (GstGLContextThreadFunc) _do_viv_direct_tex_bind_mem, &viv_tex);
++
++ return viv_tex.ret;
++}
++
+diff --git a/gst-libs/gst/gl/gstglvivdirecttexture.h b/gst-libs/gst/gl/gstglvivdirecttexture.h
+new file mode 100644
+index 0000000..fa88e1a
+--- /dev/null
++++ b/gst-libs/gst/gl/gstglvivdirecttexture.h
+@@ -0,0 +1,35 @@
++/*
++ * GStreamer
++ * Copyright (c) 2015, Freescale Semiconductor, Inc.
++ *
++ * This library is free software; you can redistribute it and/or
++ * modify it under the terms of the GNU Library General Public
++ * License as published by the Free Software Foundation; either
++ * version 2 of the License, or (at your option) any later version.
++ *
++ * This library is distributed in the hope that it will be useful,
++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
++ * Library General Public License for more details.
++ *
++ * You should have received a copy of the GNU Library General Public
++ * License along with this library; if not, write to the
++ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
++ * Boston, MA 02110-1301, USA.
++ */
++
++#ifndef __GST_GL_VIVDIRECT_H__
++#define __GST_GL_VIVDIRECT_H__
++
++#include <gst/video/video.h>
++#include <gst/gl/gstgl_fwd.h>
++
++#define GST_GL_DIRECTVIV_FORMAT "{RGBA, I420, YV12, NV12, NV21, YUY2, UYVY, BGRA, RGB16}"
++G_BEGIN_DECLS
++
++gboolean gst_is_physical_buffer (GstBuffer *buffer);
++gboolean gst_gl_viv_direct_bind_gstbuffer (GstGLContext * context, guint tex_id, GstVideoInfo * info, GstBuffer * buffer);
++
++G_END_DECLS
++
++#endif /* __GST_GL_VIVDIRECT_H__ */
+--
+1.7.9.5
+
diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
index 7e6f3ff..0728009 100644
--- a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
+++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
@@ -10,10 +10,13 @@ PACKAGECONFIG_GL_mx6sl = "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', \
base_contains('DISTRO_FEATURES', 'x11', \
'opengl', '', d), '', d)}"
-SRC_URI_append_mx6 = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
-SRC_URI_append_mx6ul = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
-SRC_URI_append_mx7 = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
+IMX_PATCHES = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch \
+ file://0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch \
+"
+SRC_URI_append_mx6 = "${IMX_PATCHES}"
+SRC_URI_append_mx6ul = "${IMX_PATCHES}"
+SRC_URI_append_mx7 = "${IMX_PATCHES}"
PACKAGE_ARCH_mx6 = "${MACHINE_SOCARCH}"
PACKAGE_ARCH_mx6ul = "${MACHINE_SOCARCH}"
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [meta-fsl-arm][PATCH v2 2/3] gstreamer1.0-plugins-bad: Support video crop for glimagesink
2016-02-02 14:01 [meta-fsl-arm][PATCH v2 0/3] Add patches for gstreamer1.0-plugins-bad 1.6 Yuqing Zhu
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance Yuqing Zhu
@ 2016-02-02 14:01 ` Yuqing Zhu
2016-03-29 7:14 ` Carlos Rafael Giani
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 3/3] gstreamer1.0-plugins-bad: Change wayland default resolution to 1024x768 Yuqing Zhu
2 siblings, 1 reply; 8+ messages in thread
From: Yuqing Zhu @ 2016-02-02 14:01 UTC (permalink / raw)
To: meta-freescale
1. Add video crop meta copy in glupload.
2. Calculate the new texture coordinate in vertices array and bind to buffer object.
3. Make glimagesink only updating vertices array when video crop meta changed.
Signed-off-by: Yuqing Zhu <b54851@freescale.com>
---
...plugin-support-video-crop-for-glimagesink.patch | 155 +++++++++++++++++++++
.../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 1 +
2 files changed, 156 insertions(+)
create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch
diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch
new file mode 100755
index 0000000..0d0ade7
--- /dev/null
+++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch
@@ -0,0 +1,155 @@
+From 1a917447c3749f0a6d3ff98b8dcbc7439b48293d Mon Sep 17 00:00:00 2001
+From: Haihua Hu <b55597@freescale.com>
+Date: Fri, 13 Nov 2015 10:51:25 +0800
+Subject: [PATCH] [glplugin] support video crop for glimagesink
+
+1.Add video crop meta copy in glupload
+2.Calculate the new texture coordinate in vertices array and bind to buffer object
+3.Make glimagesink only updating vertices array when video crop meta changed
+
+Upstream-Status: Inappropriate [i.MX specific]
+
+Signed-off-by: Haihua Hu <b55597@freescale.com>
+---
+ ext/gl/gstglimagesink.c | 53 ++++++++++++++++++++++++++++++++++++++++-
+ ext/gl/gstglimagesink.h | 3 +++
+ gst-libs/gst/gl/gstglupload.c | 10 ++++++++
+ 3 files changed, 65 insertions(+), 1 deletion(-)
+
+diff --git a/ext/gl/gstglimagesink.c b/ext/gl/gstglimagesink.c
+index 1e5dc05..6fc0f9e 100644
+--- a/ext/gl/gstglimagesink.c
++++ b/ext/gl/gstglimagesink.c
+@@ -585,6 +585,8 @@ gst_glimage_sink_init (GstGLImageSink * glimage_sink)
+ glimage_sink->handle_events = TRUE;
+ glimage_sink->ignore_alpha = TRUE;
+ glimage_sink->overlay_compositor = NULL;
++ glimage_sink->cropmeta = NULL;
++ glimage_sink->prev_cropmeta = NULL;
+
+ glimage_sink->mview_output_mode = DEFAULT_MULTIVIEW_MODE;
+ glimage_sink->mview_output_flags = DEFAULT_MULTIVIEW_FLAGS;
+@@ -1039,6 +1041,12 @@ gst_glimage_sink_change_state (GstElement * element, GstStateChange transition)
+ gst_object_unref (glimage_sink->display);
+ glimage_sink->display = NULL;
+ }
++
++ glimage_sink->cropmeta = NULL;
++ if (glimage_sink->prev_cropmeta)
++ g_slice_free(GstVideoCropMeta, glimage_sink->prev_cropmeta);
++ glimage_sink->prev_cropmeta = NULL;
++
+ break;
+ default:
+ break;
+@@ -1452,6 +1460,8 @@ gst_glimage_sink_show_frame (GstVideoSink * vsink, GstBuffer * buf)
+ GST_VIDEO_SINK_WIDTH (glimage_sink),
+ GST_VIDEO_SINK_HEIGHT (glimage_sink));
+
++ glimage_sink->cropmeta = gst_buffer_get_video_crop_meta (buf);
++
+ /* Ask the underlying window to redraw its content */
+ if (!gst_glimage_sink_redisplay (glimage_sink))
+ goto redisplay_failed;
+@@ -1638,7 +1648,7 @@ config_failed:
+ }
+
+ /* *INDENT-OFF* */
+-static const GLfloat vertices[] = {
++static GLfloat vertices[] = {
+ 1.0f, 1.0f, 0.0f, 1.0f, 0.0f,
+ -1.0f, 1.0f, 0.0f, 0.0f, 0.0f,
+ -1.0f, -1.0f, 0.0f, 0.0f, 1.0f,
+@@ -1898,6 +1908,47 @@ gst_glimage_sink_on_draw (GstGLImageSink * gl_sink)
+
+ gst_gl_shader_use (gl_sink->redisplay_shader);
+
++ if (gl_sink->cropmeta) {
++ gint width = GST_VIDEO_SINK_WIDTH (gl_sink);
++ gint height = GST_VIDEO_SINK_HEIGHT (gl_sink);
++
++ if (!gl_sink->prev_cropmeta){
++ /* Initialize the previous crop meta and set all memroy to zero */
++ gl_sink->prev_cropmeta = (GstVideoCropMeta *) g_slice_new0(GstVideoCropMeta);
++ }
++
++ /* If crop meta not equal to the previous, recalculate the vertices */
++ if (gl_sink->prev_cropmeta->x != gl_sink->cropmeta->x
++ || gl_sink->prev_cropmeta->y != gl_sink->cropmeta->y
++ || gl_sink->prev_cropmeta->width != gl_sink->cropmeta->width
++ || gl_sink->prev_cropmeta->height != gl_sink->cropmeta->height){
++
++ vertices[8] = (float)(gl_sink->cropmeta->x) / width;
++ vertices[9] = (float)(gl_sink->cropmeta->y) / height;
++
++ vertices[3] = (float)(gl_sink->cropmeta->width + gl_sink->cropmeta->x) / width;
++ vertices[4] = vertices[9];
++
++ vertices[13] = vertices[8];
++ vertices[14] = (float)(gl_sink->cropmeta->height + gl_sink->cropmeta->y) / height;
++
++ vertices[18] = vertices[3];
++ vertices[19] = vertices[14];
++
++ gl->BindBuffer (GL_ARRAY_BUFFER, gl_sink->vertex_buffer);
++ gl->BufferData (GL_ARRAY_BUFFER, 4 * 5 * sizeof (GLfloat), vertices,
++ GL_STATIC_DRAW);
++
++ gl->BindBuffer (GL_ARRAY_BUFFER, 0);
++
++ /* Store the previous crop meta */
++ gl_sink->prev_cropmeta->x = gl_sink->cropmeta->x;
++ gl_sink->prev_cropmeta->y = gl_sink->cropmeta->y;
++ gl_sink->prev_cropmeta->width = gl_sink->cropmeta->width;
++ gl_sink->prev_cropmeta->height = gl_sink->cropmeta->height;
++ }
++ }
++
+ if (gl->GenVertexArrays)
+ gl->BindVertexArray (gl_sink->vao);
+ else
+diff --git a/ext/gl/gstglimagesink.h b/ext/gl/gstglimagesink.h
+index f7b3bfb..a26ca4b 100644
+--- a/ext/gl/gstglimagesink.h
++++ b/ext/gl/gstglimagesink.h
+@@ -102,6 +102,9 @@ struct _GstGLImageSink
+ guint window_width;
+ guint window_height;
+
++ GstVideoCropMeta *cropmeta;
++ GstVideoCropMeta *prev_cropmeta;
++
+ GstVideoRectangle display_rect;
+
+ GstGLShader *redisplay_shader;
+diff --git a/gst-libs/gst/gl/gstglupload.c b/gst-libs/gst/gl/gstglupload.c
+index acaa329..0d36248 100644
+--- a/gst-libs/gst/gl/gstglupload.c
++++ b/gst-libs/gst/gl/gstglupload.c
+@@ -756,6 +756,7 @@ _physical_buffer_upload_perform(gpointer impl, GstBuffer *buffer, GstBuffer **ou
+ {
+ struct PhyBufferUpload *phyBuffer = impl;
+ GstVideoInfo *info;
++ GstVideoCropMeta *incropmeta, *outcropmeta;
+ gint n_mem;
+
+ info = &phyBuffer->upload->priv->out_info;
+@@ -774,6 +775,15 @@ _physical_buffer_upload_perform(gpointer impl, GstBuffer *buffer, GstBuffer **ou
+ gst_buffer_add_video_meta_full (*outbuf, 0,
+ GST_VIDEO_INFO_FORMAT (info), GST_VIDEO_INFO_WIDTH (info),
+ GST_VIDEO_INFO_HEIGHT (info), n_mem, info->offset, info->stride);
++ /* add video crop meta to out buffer if need */
++ incropmeta = gst_buffer_get_video_crop_meta(buffer);
++ if(incropmeta){
++ outcropmeta = gst_buffer_add_video_crop_meta(*outbuf);
++ outcropmeta->x = incropmeta->x;
++ outcropmeta->y = incropmeta->y;
++ outcropmeta->width = incropmeta->width;
++ outcropmeta->height = incropmeta->height;
++ }
+
+ return GST_GL_UPLOAD_DONE;
+ }
+--
+1.7.9.5
+
diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
index 0728009..9ce22eb 100644
--- a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
+++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
@@ -12,6 +12,7 @@ PACKAGECONFIG_GL_mx6sl = "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', \
IMX_PATCHES = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch \
file://0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch \
+ file://0003-glplugin-support-video-crop-for-glimagesink.patch \
"
SRC_URI_append_mx6 = "${IMX_PATCHES}"
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [meta-fsl-arm][PATCH v2 3/3] gstreamer1.0-plugins-bad: Change wayland default resolution to 1024x768
2016-02-02 14:01 [meta-fsl-arm][PATCH v2 0/3] Add patches for gstreamer1.0-plugins-bad 1.6 Yuqing Zhu
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance Yuqing Zhu
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 2/3] gstreamer1.0-plugins-bad: Support video crop for glimagesink Yuqing Zhu
@ 2016-02-02 14:01 ` Yuqing Zhu
2016-03-29 7:07 ` Carlos Rafael Giani
2 siblings, 1 reply; 8+ messages in thread
From: Yuqing Zhu @ 2016-02-02 14:01 UTC (permalink / raw)
To: meta-freescale
In our release, the default display is full screen in LVDS.
The 1024x768 resolution will be better in testing for our
release platform.
Signed-off-by: Yuqing Zhu <b54851@freescale.com>
---
...in-Change-wayland-default-res-to-1024x768.patch | 41 ++++++++++++++++++++++
.../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 1 +
2 files changed, 42 insertions(+)
create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch
diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch
new file mode 100755
index 0000000..49a0a9d
--- /dev/null
+++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch
@@ -0,0 +1,41 @@
+From decc71a707d3b3596888d30a4c7ca25b57a686b2 Mon Sep 17 00:00:00 2001
+From: Jian <Jian.Li@freescale.com>
+Date: Tue, 17 Nov 2015 14:39:07 +0800
+Subject: [PATCH 2/3] [glplugin] Change wayland default res to 1024x768
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+Change wayland backend default window resolution
+from 320x240 to 1024x768
+
+Upstream-Status: Inappropriate [i.MX specific]
+
+Signed-off-by: Jian <Jian.Li@freescale.com>
+---
+ gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
+
+diff --git a/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c b/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c
+index b34ad42..b557daa 100644
+--- a/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c
++++ b/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c
+@@ -305,13 +305,13 @@ create_surfaces (GstGLWindowWaylandEGL * window_egl)
+ if (window_egl->window.window_width > 0)
+ width = window_egl->window.window_width;
+ else
+- width = 320;
++ width = 1024;
+ window_egl->window.window_width = width;
+
+ if (window_egl->window.window_height > 0)
+ height = window_egl->window.window_height;
+ else
+- height = 240;
++ height = 768;
+ window_egl->window.window_height = height;
+
+ if (!window_egl->window.native) {
+--
+1.7.9.5
+
diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
index 9ce22eb..ea66be9 100644
--- a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
+++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
@@ -13,6 +13,7 @@ PACKAGECONFIG_GL_mx6sl = "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', \
IMX_PATCHES = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch \
file://0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch \
file://0003-glplugin-support-video-crop-for-glimagesink.patch \
+ file://0004-glplugin-Change-wayland-default-res-to-1024x768.patch \
"
SRC_URI_append_mx6 = "${IMX_PATCHES}"
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [meta-fsl-arm][PATCH v2 3/3] gstreamer1.0-plugins-bad: Change wayland default resolution to 1024x768
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 3/3] gstreamer1.0-plugins-bad: Change wayland default resolution to 1024x768 Yuqing Zhu
@ 2016-03-29 7:07 ` Carlos Rafael Giani
0 siblings, 0 replies; 8+ messages in thread
From: Carlos Rafael Giani @ 2016-03-29 7:07 UTC (permalink / raw)
To: Yuqing Zhu, meta-freescale
This gets an Ack from me. It doesn't break anything, and is truly i.MX
specific. There may be i.MX devices with screens smaller than 1024x768,
though to me this seems to be the minority.
On 2016-02-02 15:01, Yuqing Zhu wrote:
> In our release, the default display is full screen in LVDS.
> The 1024x768 resolution will be better in testing for our
> release platform.
>
> Signed-off-by: Yuqing Zhu <b54851@freescale.com>
> ---
> ...in-Change-wayland-default-res-to-1024x768.patch | 41 ++++++++++++++++++++++
> .../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 1 +
> 2 files changed, 42 insertions(+)
> create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch
>
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch
> new file mode 100755
> index 0000000..49a0a9d
> --- /dev/null
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0004-glplugin-Change-wayland-default-res-to-1024x768.patch
> @@ -0,0 +1,41 @@
> +From decc71a707d3b3596888d30a4c7ca25b57a686b2 Mon Sep 17 00:00:00 2001
> +From: Jian <Jian.Li@freescale.com>
> +Date: Tue, 17 Nov 2015 14:39:07 +0800
> +Subject: [PATCH 2/3] [glplugin] Change wayland default res to 1024x768
> +MIME-Version: 1.0
> +Content-Type: text/plain; charset=UTF-8
> +Content-Transfer-Encoding: 8bit
> +
> +Change wayland backend default window resolution
> +from 320x240 to 1024x768
> +
> +Upstream-Status: Inappropriate [i.MX specific]
> +
> +Signed-off-by: Jian <Jian.Li@freescale.com>
> +---
> + gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c | 4 ++--
> + 1 file changed, 2 insertions(+), 2 deletions(-)
> +
> +diff --git a/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c b/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c
> +index b34ad42..b557daa 100644
> +--- a/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c
> ++++ b/gst-libs/gst/gl/wayland/gstglwindow_wayland_egl.c
> +@@ -305,13 +305,13 @@ create_surfaces (GstGLWindowWaylandEGL * window_egl)
> + if (window_egl->window.window_width > 0)
> + width = window_egl->window.window_width;
> + else
> +- width = 320;
> ++ width = 1024;
> + window_egl->window.window_width = width;
> +
> + if (window_egl->window.window_height > 0)
> + height = window_egl->window.window_height;
> + else
> +- height = 240;
> ++ height = 768;
> + window_egl->window.window_height = height;
> +
> + if (!window_egl->window.native) {
> +--
> +1.7.9.5
> +
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> index 9ce22eb..ea66be9 100644
> --- a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> @@ -13,6 +13,7 @@ PACKAGECONFIG_GL_mx6sl = "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', \
> IMX_PATCHES = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch \
> file://0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch \
> file://0003-glplugin-support-video-crop-for-glimagesink.patch \
> + file://0004-glplugin-Change-wayland-default-res-to-1024x768.patch \
> "
>
> SRC_URI_append_mx6 = "${IMX_PATCHES}"
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-fsl-arm][PATCH v2 2/3] gstreamer1.0-plugins-bad: Support video crop for glimagesink
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 2/3] gstreamer1.0-plugins-bad: Support video crop for glimagesink Yuqing Zhu
@ 2016-03-29 7:14 ` Carlos Rafael Giani
0 siblings, 0 replies; 8+ messages in thread
From: Carlos Rafael Giani @ 2016-03-29 7:14 UTC (permalink / raw)
To: Yuqing Zhu, meta-freescale
This seems to be quite useful to me. But it is not i.MX specific - I
could for example even use this on the PC.
However, modifying the global vertices array is not good, for two reasons:
1. This will not work well in a multithreaded environment (with multiple
glimagesink instances)
2. The vertex data is not reset properly when the state change goes back
to READY
So, instead, I'd keep this array const, and instead copy it & modify and
upload the local copy. Crop metas are not updated so often, so the costs
for a copy are negligible.
Once this is fixed, I strongly recommend to put this on GStreamer's
bugzilla.
On 2016-02-02 15:01, Yuqing Zhu wrote:
> 1. Add video crop meta copy in glupload.
> 2. Calculate the new texture coordinate in vertices array and bind to buffer object.
> 3. Make glimagesink only updating vertices array when video crop meta changed.
>
> Signed-off-by: Yuqing Zhu <b54851@freescale.com>
> ---
> ...plugin-support-video-crop-for-glimagesink.patch | 155 +++++++++++++++++++++
> .../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 1 +
> 2 files changed, 156 insertions(+)
> create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch
>
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch
> new file mode 100755
> index 0000000..0d0ade7
> --- /dev/null
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0003-glplugin-support-video-crop-for-glimagesink.patch
> @@ -0,0 +1,155 @@
> +From 1a917447c3749f0a6d3ff98b8dcbc7439b48293d Mon Sep 17 00:00:00 2001
> +From: Haihua Hu <b55597@freescale.com>
> +Date: Fri, 13 Nov 2015 10:51:25 +0800
> +Subject: [PATCH] [glplugin] support video crop for glimagesink
> +
> +1.Add video crop meta copy in glupload
> +2.Calculate the new texture coordinate in vertices array and bind to buffer object
> +3.Make glimagesink only updating vertices array when video crop meta changed
> +
> +Upstream-Status: Inappropriate [i.MX specific]
> +
> +Signed-off-by: Haihua Hu <b55597@freescale.com>
> +---
> + ext/gl/gstglimagesink.c | 53 ++++++++++++++++++++++++++++++++++++++++-
> + ext/gl/gstglimagesink.h | 3 +++
> + gst-libs/gst/gl/gstglupload.c | 10 ++++++++
> + 3 files changed, 65 insertions(+), 1 deletion(-)
> +
> +diff --git a/ext/gl/gstglimagesink.c b/ext/gl/gstglimagesink.c
> +index 1e5dc05..6fc0f9e 100644
> +--- a/ext/gl/gstglimagesink.c
> ++++ b/ext/gl/gstglimagesink.c
> +@@ -585,6 +585,8 @@ gst_glimage_sink_init (GstGLImageSink * glimage_sink)
> + glimage_sink->handle_events = TRUE;
> + glimage_sink->ignore_alpha = TRUE;
> + glimage_sink->overlay_compositor = NULL;
> ++ glimage_sink->cropmeta = NULL;
> ++ glimage_sink->prev_cropmeta = NULL;
> +
> + glimage_sink->mview_output_mode = DEFAULT_MULTIVIEW_MODE;
> + glimage_sink->mview_output_flags = DEFAULT_MULTIVIEW_FLAGS;
> +@@ -1039,6 +1041,12 @@ gst_glimage_sink_change_state (GstElement * element, GstStateChange transition)
> + gst_object_unref (glimage_sink->display);
> + glimage_sink->display = NULL;
> + }
> ++
> ++ glimage_sink->cropmeta = NULL;
> ++ if (glimage_sink->prev_cropmeta)
> ++ g_slice_free(GstVideoCropMeta, glimage_sink->prev_cropmeta);
> ++ glimage_sink->prev_cropmeta = NULL;
> ++
> + break;
> + default:
> + break;
> +@@ -1452,6 +1460,8 @@ gst_glimage_sink_show_frame (GstVideoSink * vsink, GstBuffer * buf)
> + GST_VIDEO_SINK_WIDTH (glimage_sink),
> + GST_VIDEO_SINK_HEIGHT (glimage_sink));
> +
> ++ glimage_sink->cropmeta = gst_buffer_get_video_crop_meta (buf);
> ++
> + /* Ask the underlying window to redraw its content */
> + if (!gst_glimage_sink_redisplay (glimage_sink))
> + goto redisplay_failed;
> +@@ -1638,7 +1648,7 @@ config_failed:
> + }
> +
> + /* *INDENT-OFF* */
> +-static const GLfloat vertices[] = {
> ++static GLfloat vertices[] = {
> + 1.0f, 1.0f, 0.0f, 1.0f, 0.0f,
> + -1.0f, 1.0f, 0.0f, 0.0f, 0.0f,
> + -1.0f, -1.0f, 0.0f, 0.0f, 1.0f,
> +@@ -1898,6 +1908,47 @@ gst_glimage_sink_on_draw (GstGLImageSink * gl_sink)
> +
> + gst_gl_shader_use (gl_sink->redisplay_shader);
> +
> ++ if (gl_sink->cropmeta) {
> ++ gint width = GST_VIDEO_SINK_WIDTH (gl_sink);
> ++ gint height = GST_VIDEO_SINK_HEIGHT (gl_sink);
> ++
> ++ if (!gl_sink->prev_cropmeta){
> ++ /* Initialize the previous crop meta and set all memroy to zero */
> ++ gl_sink->prev_cropmeta = (GstVideoCropMeta *) g_slice_new0(GstVideoCropMeta);
> ++ }
> ++
> ++ /* If crop meta not equal to the previous, recalculate the vertices */
> ++ if (gl_sink->prev_cropmeta->x != gl_sink->cropmeta->x
> ++ || gl_sink->prev_cropmeta->y != gl_sink->cropmeta->y
> ++ || gl_sink->prev_cropmeta->width != gl_sink->cropmeta->width
> ++ || gl_sink->prev_cropmeta->height != gl_sink->cropmeta->height){
> ++
> ++ vertices[8] = (float)(gl_sink->cropmeta->x) / width;
> ++ vertices[9] = (float)(gl_sink->cropmeta->y) / height;
> ++
> ++ vertices[3] = (float)(gl_sink->cropmeta->width + gl_sink->cropmeta->x) / width;
> ++ vertices[4] = vertices[9];
> ++
> ++ vertices[13] = vertices[8];
> ++ vertices[14] = (float)(gl_sink->cropmeta->height + gl_sink->cropmeta->y) / height;
> ++
> ++ vertices[18] = vertices[3];
> ++ vertices[19] = vertices[14];
> ++
> ++ gl->BindBuffer (GL_ARRAY_BUFFER, gl_sink->vertex_buffer);
> ++ gl->BufferData (GL_ARRAY_BUFFER, 4 * 5 * sizeof (GLfloat), vertices,
> ++ GL_STATIC_DRAW);
> ++
> ++ gl->BindBuffer (GL_ARRAY_BUFFER, 0);
> ++
> ++ /* Store the previous crop meta */
> ++ gl_sink->prev_cropmeta->x = gl_sink->cropmeta->x;
> ++ gl_sink->prev_cropmeta->y = gl_sink->cropmeta->y;
> ++ gl_sink->prev_cropmeta->width = gl_sink->cropmeta->width;
> ++ gl_sink->prev_cropmeta->height = gl_sink->cropmeta->height;
> ++ }
> ++ }
> ++
> + if (gl->GenVertexArrays)
> + gl->BindVertexArray (gl_sink->vao);
> + else
> +diff --git a/ext/gl/gstglimagesink.h b/ext/gl/gstglimagesink.h
> +index f7b3bfb..a26ca4b 100644
> +--- a/ext/gl/gstglimagesink.h
> ++++ b/ext/gl/gstglimagesink.h
> +@@ -102,6 +102,9 @@ struct _GstGLImageSink
> + guint window_width;
> + guint window_height;
> +
> ++ GstVideoCropMeta *cropmeta;
> ++ GstVideoCropMeta *prev_cropmeta;
> ++
> + GstVideoRectangle display_rect;
> +
> + GstGLShader *redisplay_shader;
> +diff --git a/gst-libs/gst/gl/gstglupload.c b/gst-libs/gst/gl/gstglupload.c
> +index acaa329..0d36248 100644
> +--- a/gst-libs/gst/gl/gstglupload.c
> ++++ b/gst-libs/gst/gl/gstglupload.c
> +@@ -756,6 +756,7 @@ _physical_buffer_upload_perform(gpointer impl, GstBuffer *buffer, GstBuffer **ou
> + {
> + struct PhyBufferUpload *phyBuffer = impl;
> + GstVideoInfo *info;
> ++ GstVideoCropMeta *incropmeta, *outcropmeta;
> + gint n_mem;
> +
> + info = &phyBuffer->upload->priv->out_info;
> +@@ -774,6 +775,15 @@ _physical_buffer_upload_perform(gpointer impl, GstBuffer *buffer, GstBuffer **ou
> + gst_buffer_add_video_meta_full (*outbuf, 0,
> + GST_VIDEO_INFO_FORMAT (info), GST_VIDEO_INFO_WIDTH (info),
> + GST_VIDEO_INFO_HEIGHT (info), n_mem, info->offset, info->stride);
> ++ /* add video crop meta to out buffer if need */
> ++ incropmeta = gst_buffer_get_video_crop_meta(buffer);
> ++ if(incropmeta){
> ++ outcropmeta = gst_buffer_add_video_crop_meta(*outbuf);
> ++ outcropmeta->x = incropmeta->x;
> ++ outcropmeta->y = incropmeta->y;
> ++ outcropmeta->width = incropmeta->width;
> ++ outcropmeta->height = incropmeta->height;
> ++ }
> +
> + return GST_GL_UPLOAD_DONE;
> + }
> +--
> +1.7.9.5
> +
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> index 0728009..9ce22eb 100644
> --- a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> @@ -12,6 +12,7 @@ PACKAGECONFIG_GL_mx6sl = "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', \
>
> IMX_PATCHES = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch \
> file://0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch \
> + file://0003-glplugin-support-video-crop-for-glimagesink.patch \
> "
>
> SRC_URI_append_mx6 = "${IMX_PATCHES}"
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance Yuqing Zhu
@ 2016-03-29 7:32 ` Carlos Rafael Giani
2016-03-31 8:03 ` Yuqing Zhu
0 siblings, 1 reply; 8+ messages in thread
From: Carlos Rafael Giani @ 2016-03-29 7:32 UTC (permalink / raw)
To: Yuqing Zhu, meta-freescale
Generally this patch seems OK. There are some places that need
clarification.
@@ -231,6 +233,18 @@ gst_gl_upload_element_prepare_output_buffer
(GstBaseTransform * bt,
return GST_FLOW_ERROR;
}
+ /* check if the output buffer's meta format equals to the caps format */
+ GstVideoMeta *vmeta = gst_buffer_get_video_meta(*outbuf);
+ gst_video_info_from_caps (&info, upload->out_caps);
+ if(vmeta->format != GST_VIDEO_INFO_FORMAT (&info))
+ {
+ caps = gst_caps_copy(upload->out_caps);
+ gst_caps_set_simple (caps, "format", G_TYPE_STRING,
+ gst_video_format_to_string (vmeta->format), NULL);
+ gst_caps_replace (&upload->out_caps, caps);
+ gst_base_transform_update_src_caps (upload, caps);
+ }
+
This is to ensure that the output format always corresponds to the
actual video frame's format? This is because the direct textures can
handle pixel format conversions automatically, right? But what if the
meta contains a format that is not supported by the direct texture?
(Y42B for example, which is not uncommon with USB webcams)
@@ -998,7 +1110,7 @@ gst_gl_upload_transform_caps (GstGLContext *
context, GstPadDirection direction,
if (filter) {
- result = gst_caps_intersect_full (filter, tmp,
GST_CAPS_INTERSECT_FIRST);
+ result = gst_caps_intersect_full (tmp, filter,
GST_CAPS_INTERSECT_FIRST);
gst_caps_unref (tmp);
} else {
result = tmp;
Why is this done?
@@ -896,6 +896,11 @@ gst_glimage_sink_query (GstBaseSink * bsink,
GstQuery * query)
res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
break;
}
+ case GST_QUERY_ALLOCATION:
+ {
+ res = gst_glimage_sink_propose_allocation(bsink, query);
+ break;
+ }
default:
res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
break;
This one is interesting. Without this, there will be a memory copy?
Seems counter intuitive for glimagesink to do this.
Also, I think a similar patch was being used in a separate project. I
will look it up. Perhaps these can be merged.
On 2016-02-02 15:01, Yuqing Zhu wrote:
> 1. Add a physical buffer uploader in glupload plugin and using viv direct
> texture to bind physical continious buffer with texture to avoid memory
> copy from videobuffer to texture to gain good performance.
>
> 2. Reduce glimagesink load latency by override glimagesink ALLOCATION query to
> avoid memory copy.
>
> Signed-off-by: Yuqing Zhu <b54851@freescale.com>
> ---
> ...imagesink-to-improve-playback-performance.patch | 454 +++++++++++++++++++++
> .../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 9 +-
> 2 files changed, 460 insertions(+), 3 deletions(-)
> create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
>
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
> new file mode 100755
> index 0000000..2739355
> --- /dev/null
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
> @@ -0,0 +1,454 @@
> +From 587db892fdaac18780c8024e5a976ea182f36ff4 Mon Sep 17 00:00:00 2001
> +From: Haihua Hu <b55597@freescale.com>
> +Date: Tue, 13 Oct 2015 09:33:54 +0800
> +Subject: [PATCH 1/2] [glplugin] Add directviv to glimagesink to improve
> + playback performance
> +MIME-Version: 1.0
> +Content-Type: text/plain; charset=UTF-8
> +Content-Transfer-Encoding: 8bit
> +
> +1.Add a physical buffer uploader in glupload plugin and using viv direct
> + texture to bind physical continious buffer with texture to avoid memory
> + copy from videobuffer to texture to gain good performance.
> +2.Reduce glimagesink load latency by override glimagesink ALLOCATION query to
> + avoid memory copy.
> +
> +Upstream-Status: Inappropriate [i.MX specific]
> +
> +Signed-off-by: Haihua Hu <b55597@freescale.com>
> +---
> + ext/gl/gstglimagesink.c | 5 ++
> + ext/gl/gstgluploadelement.c | 14 +++
> + gst-libs/gst/gl/Makefile.am | 2 +
> + gst-libs/gst/gl/gstglupload.c | 118 ++++++++++++++++++++++++-
> + gst-libs/gst/gl/gstglvivdirecttexture.c | 143 +++++++++++++++++++++++++++++++
> + gst-libs/gst/gl/gstglvivdirecttexture.h | 35 ++++++++
> + 6 files changed, 314 insertions(+), 3 deletions(-)
> + create mode 100644 gst-libs/gst/gl/gstglvivdirecttexture.c
> + create mode 100644 gst-libs/gst/gl/gstglvivdirecttexture.h
> +
> +diff --git a/ext/gl/gstglimagesink.c b/ext/gl/gstglimagesink.c
> +index f629df7..1e5dc05 100644
> +--- a/ext/gl/gstglimagesink.c
> ++++ b/ext/gl/gstglimagesink.c
> +@@ -896,6 +896,11 @@ gst_glimage_sink_query (GstBaseSink * bsink, GstQuery * query)
> + res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
> + break;
> + }
> ++ case GST_QUERY_ALLOCATION:
> ++ {
> ++ res = gst_glimage_sink_propose_allocation(bsink, query);
> ++ break;
> ++ }
> + default:
> + res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
> + break;
> +diff --git a/ext/gl/gstgluploadelement.c b/ext/gl/gstgluploadelement.c
> +index 83ff24c..e74da3a 100644
> +--- a/ext/gl/gstgluploadelement.c
> ++++ b/ext/gl/gstgluploadelement.c
> +@@ -208,6 +208,8 @@ gst_gl_upload_element_prepare_output_buffer (GstBaseTransform * bt,
> + GstGLUploadElement *upload = GST_GL_UPLOAD_ELEMENT (bt);
> + GstGLUploadReturn ret;
> + GstBaseTransformClass *bclass;
> ++ GstVideoInfo info;
> ++ GstCaps *caps;
> +
> + bclass = GST_BASE_TRANSFORM_GET_CLASS (bt);
> +
> +@@ -231,6 +233,18 @@ gst_gl_upload_element_prepare_output_buffer (GstBaseTransform * bt,
> + return GST_FLOW_ERROR;
> + }
> +
> ++ /* check if the output buffer's meta format equals to the caps format */
> ++ GstVideoMeta *vmeta = gst_buffer_get_video_meta(*outbuf);
> ++ gst_video_info_from_caps (&info, upload->out_caps);
> ++ if(vmeta->format != GST_VIDEO_INFO_FORMAT (&info))
> ++ {
> ++ caps = gst_caps_copy(upload->out_caps);
> ++ gst_caps_set_simple (caps, "format", G_TYPE_STRING,
> ++ gst_video_format_to_string (vmeta->format), NULL);
> ++ gst_caps_replace (&upload->out_caps, caps);
> ++ gst_base_transform_update_src_caps (upload, caps);
> ++ }
> ++
> + /* basetransform doesn't unref if they're the same */
> + if (buffer == *outbuf)
> + gst_buffer_unref (*outbuf);
> +diff --git a/gst-libs/gst/gl/Makefile.am b/gst-libs/gst/gl/Makefile.am
> +index 36f3bf7..402e37a 100644
> +--- a/gst-libs/gst/gl/Makefile.am
> ++++ b/gst-libs/gst/gl/Makefile.am
> +@@ -30,6 +30,7 @@ libgstgl_@GST_API_VERSION@_la_SOURCES = \
> + gstglsyncmeta.c \
> + gstglviewconvert.c \
> + gstgloverlaycompositor.c \
> ++ gstglvivdirecttexture.c \
> + utils/opengl_versions.h
> +
> + libgstgl_@GST_API_VERSION@includedir = $(includedir)/gstreamer-@GST_API_VERSION@/gst/gl
> +@@ -56,6 +57,7 @@ libgstgl_@GST_API_VERSION@include_HEADERS = \
> + gstglviewconvert.h \
> + gstgloverlaycompositor.h \
> + gstgl_fwd.h \
> ++ gstglvivdirecttexture.h \
> + gl.h
> +
> + libgstgl_@GST_API_VERSION@_la_LIBADD = \
> +diff --git a/gst-libs/gst/gl/gstglupload.c b/gst-libs/gst/gl/gstglupload.c
> +index 0addca1..acaa329 100644
> +--- a/gst-libs/gst/gl/gstglupload.c
> ++++ b/gst-libs/gst/gl/gstglupload.c
> +@@ -23,6 +23,7 @@
> + #endif
> +
> + #include <stdio.h>
> ++#include <gst/gl/gstglvivdirecttexture.h>
> +
> + #include "gl.h"
> + #include "gstglupload.h"
> +@@ -47,7 +48,7 @@
> + #define USING_GLES2(context) (gst_gl_context_check_gl_version (context, GST_GL_API_GLES2, 2, 0))
> + #define USING_GLES3(context) (gst_gl_context_check_gl_version (context, GST_GL_API_GLES2, 3, 0))
> +
> +-GST_DEBUG_CATEGORY_STATIC (gst_gl_upload_debug);
> ++GST_DEBUG_CATEGORY (gst_gl_upload_debug);
> + #define GST_CAT_DEFAULT gst_gl_upload_debug
> +
> + #define DEBUG_INIT \
> +@@ -698,6 +699,114 @@ static const UploadMethod _upload_meta_upload = {
> + &_upload_meta_upload_free
> + };
> +
> ++struct PhyBufferUpload
> ++{
> ++ GstGLUpload *upload;
> ++};
> ++
> ++static gpointer
> ++_physical_buffer_upload_new(GstGLUpload *upload)
> ++{
> ++ struct PhyBufferUpload *phybuffer = g_new0 (struct PhyBufferUpload, 1);
> ++
> ++ phybuffer->upload = upload;
> ++
> ++ return phybuffer;
> ++}
> ++
> ++static GstCaps *
> ++_physical_buffer_upload_transform_caps(GstGLContext *context,
> ++ GstPadDirection direction, GstCaps *caps)
> ++{
> ++ GstCaps *ret;
> ++
> ++ if (direction == GST_PAD_SINK) {
> ++ ret = _set_caps_features (caps, GST_CAPS_FEATURE_MEMORY_GL_MEMORY);
> ++ gst_caps_set_simple (ret, "format", G_TYPE_STRING, "RGBA", NULL);
> ++ } else {
> ++ ret = gst_caps_from_string (GST_VIDEO_CAPS_MAKE_WITH_FEATURES
> ++ (GST_CAPS_FEATURE_MEMORY_SYSTEM_MEMORY,GST_GL_DIRECTVIV_FORMAT));
> ++ }
> ++
> ++ return ret;
> ++}
> ++
> ++static gboolean
> ++_physical_buffer_upload_accept(gpointer impl, GstBuffer *buffer,
> ++ GstCaps *in_caps, GstCaps *out_caps)
> ++{
> ++ GstCapsFeatures *features;
> ++
> ++ features = gst_caps_get_features (out_caps, 0);
> ++ if (!gst_caps_features_contains (features, GST_CAPS_FEATURE_MEMORY_GL_MEMORY))
> ++ return FALSE;
> ++
> ++ return gst_is_physical_buffer(buffer);
> ++}
> ++
> ++static void
> ++_physical_buffer_upload_propose_allocation(gpointer impl, GstQuery *decide_query,
> ++ GstQuery *query)
> ++{
> ++ gst_query_add_allocation_meta (query, GST_VIDEO_META_API_TYPE, 0);
> ++}
> ++
> ++static GstGLUploadReturn
> ++_physical_buffer_upload_perform(gpointer impl, GstBuffer *buffer, GstBuffer **outbuf)
> ++{
> ++ struct PhyBufferUpload *phyBuffer = impl;
> ++ GstVideoInfo *info;
> ++ gint n_mem;
> ++
> ++ info = &phyBuffer->upload->priv->out_info;
> ++ n_mem = GST_VIDEO_INFO_N_PLANES (info);
> ++ GST_LOG_OBJECT (phyBuffer->upload, "Attempting viv direct upload");
> ++ *outbuf = gst_buffer_new ();
> ++ gst_gl_memory_setup_buffer (phyBuffer->upload->context,
> ++ NULL, info, NULL, *outbuf);
> ++
> ++ GstGLMemory *out_gl_mem =
> ++ (GstGLMemory *) gst_buffer_peek_memory (*outbuf, 0);
> ++
> ++ gst_gl_viv_direct_bind_gstbuffer(phyBuffer->upload->context, out_gl_mem->tex_id,
> ++ &phyBuffer->upload->priv->in_info, buffer);
> ++
> ++ gst_buffer_add_video_meta_full (*outbuf, 0,
> ++ GST_VIDEO_INFO_FORMAT (info), GST_VIDEO_INFO_WIDTH (info),
> ++ GST_VIDEO_INFO_HEIGHT (info), n_mem, info->offset, info->stride);
> ++
> ++ return GST_GL_UPLOAD_DONE;
> ++}
> ++
> ++static void
> ++_physical_buffer_upload_release(gpointer impl, GstBuffer *buffer)
> ++{
> ++}
> ++
> ++static void
> ++_physical_buffer_upload_free(gpointer impl)
> ++{
> ++ struct PhyBufferUpload *phyBuffer = impl;
> ++
> ++ g_free(phyBuffer);
> ++}
> ++
> ++static GstStaticCaps _physical_buffer_upload_caps =
> ++GST_STATIC_CAPS (GST_VIDEO_CAPS_MAKE (GST_GL_DIRECTVIV_FORMAT));
> ++
> ++static const UploadMethod _physical_buffer_upload = {
> ++ "Physical buffer",
> ++ 0,
> ++ &_physical_buffer_upload_caps,
> ++ &_physical_buffer_upload_new,
> ++ &_physical_buffer_upload_transform_caps,
> ++ &_physical_buffer_upload_accept,
> ++ &_physical_buffer_upload_propose_allocation,
> ++ &_physical_buffer_upload_perform,
> ++ &_physical_buffer_upload_release,
> ++ &_physical_buffer_upload_free
> ++};
> ++
> + struct RawUploadFrame
> + {
> + gint ref_count;
> +@@ -813,6 +922,7 @@ _raw_data_upload_perform (gpointer impl, GstBuffer * buffer,
> + struct RawUpload *raw = impl;
> + int i;
> + GstVideoInfo *in_info = &raw->upload->priv->in_info;
> ++
> + guint max_planes = GST_VIDEO_INFO_N_PLANES (in_info);
> +
> + /* Support stereo views for separated multiview mode */
> +@@ -870,7 +980,9 @@ static const UploadMethod *upload_methods[] = { &_gl_memory_upload,
> + #if GST_GL_HAVE_PLATFORM_EGL
> + &_egl_image_upload,
> + #endif
> +- &_upload_meta_upload, &_raw_data_upload
> ++ &_upload_meta_upload,
> ++ &_physical_buffer_upload,
> ++ &_raw_data_upload
> + };
> +
> + static GMutex upload_global_lock;
> +@@ -998,7 +1110,7 @@ gst_gl_upload_transform_caps (GstGLContext * context, GstPadDirection direction,
> +
> +
> + if (filter) {
> +- result = gst_caps_intersect_full (filter, tmp, GST_CAPS_INTERSECT_FIRST);
> ++ result = gst_caps_intersect_full (tmp, filter, GST_CAPS_INTERSECT_FIRST);
> + gst_caps_unref (tmp);
> + } else {
> + result = tmp;
> +diff --git a/gst-libs/gst/gl/gstglvivdirecttexture.c b/gst-libs/gst/gl/gstglvivdirecttexture.c
> +new file mode 100644
> +index 0000000..c19b617
> +--- /dev/null
> ++++ b/gst-libs/gst/gl/gstglvivdirecttexture.c
> +@@ -0,0 +1,143 @@
> ++/*
> ++ * GStreamer
> ++ * Copyright (c) 2015, Freescale Semiconductor, Inc.
> ++ *
> ++ * This library is free software; you can redistribute it and/or
> ++ * modify it under the terms of the GNU Library General Public
> ++ * License as published by the Free Software Foundation; either
> ++ * version 2 of the License, or (at your option) any later version.
> ++ *
> ++ * This library is distributed in the hope that it will be useful,
> ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
> ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> ++ * Library General Public License for more details.
> ++ *
> ++ * You should have received a copy of the GNU Library General Public
> ++ * License along with this library; if not, write to the
> ++ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
> ++ * Boston, MA 02110-1301, USA.
> ++ */
> ++
> ++#ifdef HAVE_CONFIG_H
> ++#include "config.h"
> ++#endif
> ++
> ++#include "gl.h"
> ++
> ++GST_DEBUG_CATEGORY_EXTERN (gst_gl_upload_debug);
> ++#define GST_CAT_DEFAULT gst_gl_upload_debug
> ++
> ++typedef struct {
> ++ guint tex_id;
> ++ guint w;
> ++ guint h;
> ++ guint fmt;
> ++ void *vaddr;
> ++ guint paddr;
> ++ gboolean ret;
> ++} GstVivDirectTexture;
> ++
> ++gboolean
> ++gst_is_physical_buffer (GstBuffer *buffer)
> ++{
> ++
> ++ GstMemory *mem;
> ++
> ++ mem = gst_buffer_peek_memory (buffer, 0);
> ++ if (!mem->allocator)
> ++ return FALSE;
> ++
> ++ return g_type_check_instance_is_a (mem->allocator, g_type_from_name("GstAllocatorPhyMem"));
> ++}
> ++
> ++static void
> ++_do_viv_direct_tex_bind_mem (GstGLContext * context, GstVivDirectTexture * viv_tex)
> ++{
> ++ GST_DEBUG ("viv direct upload, tex_id %d, fmt: %d, res: (%dx%d)", viv_tex->tex_id, viv_tex->fmt, viv_tex->w, viv_tex->h);
> ++ GST_DEBUG ("Physical memory buffer, vaddr: %p, paddr: %p", viv_tex->vaddr, viv_tex->paddr);
> ++
> ++ glBindTexture (GL_TEXTURE_2D, viv_tex->tex_id);
> ++ glTexDirectVIVMap (GL_TEXTURE_2D, viv_tex->w, viv_tex->h, viv_tex->fmt, &viv_tex->vaddr, &viv_tex->paddr);
> ++ glTexDirectInvalidateVIV (GL_TEXTURE_2D);
> ++ viv_tex->ret = TRUE;
> ++
> ++ return;
> ++}
> ++
> ++gboolean
> ++gst_gl_viv_direct_bind_gstbuffer (GstGLContext * context, guint tex_id, GstVideoInfo * info, GstBuffer * buffer)
> ++{
> ++ typedef struct {
> ++ guint8 *vaddr;
> ++ guint8 *paddr;
> ++ guint8 *caddr;
> ++ gsize size;
> ++ gpointer *user_data;
> ++ } PhyMemBlock;
> ++ //Note: structure PhyMemBlock is copied from gst1.0-fsl-plugin/libs/allocator/gstallocatorphymem.h
> ++
> ++ typedef struct {
> ++ GstMemory mem;
> ++ guint8 *vaddr;
> ++ guint8 *paddr;
> ++ PhyMemBlock block;
> ++ } GstMemoryPhy;
> ++ //Note: structure GstMemoryPhy is copied from gst1.0-fsl-plugin/libs/allocator/gstallocatorphymem.c
> ++
> ++ GstMemory *mem = gst_buffer_peek_memory (buffer, 0);
> ++ GstMemoryPhy *memphy = (GstMemoryPhy*) mem;
> ++ PhyMemBlock *memblk = &memphy->block;
> ++
> ++ GstVideoFormat fmt = GST_VIDEO_INFO_FORMAT (info);
> ++ gint width, height;
> ++ GstVideoMeta *vmeta = gst_buffer_get_video_meta (buffer);
> ++ if (vmeta && (fmt == GST_VIDEO_FORMAT_I420 || fmt == GST_VIDEO_FORMAT_NV12)) {
> ++ width = vmeta->stride[0];
> ++ height = vmeta->offset[1] / width;
> ++ }
> ++ else {
> ++ width = GST_VIDEO_INFO_WIDTH (info);
> ++ height = GST_VIDEO_INFO_HEIGHT (info);
> ++ }
> ++
> ++ guint viv_fmt;
> ++ switch (fmt) {
> ++ case GST_VIDEO_FORMAT_I420:
> ++ viv_fmt = GL_VIV_I420;
> ++ break;
> ++ case GST_VIDEO_FORMAT_YV12:
> ++ viv_fmt = GL_VIV_YV12;
> ++ break;
> ++ case GST_VIDEO_FORMAT_NV12:
> ++ viv_fmt = GL_VIV_NV12;
> ++ break;
> ++ case GST_VIDEO_FORMAT_NV21:
> ++ viv_fmt = GL_VIV_NV21;
> ++ break;
> ++ case GST_VIDEO_FORMAT_YUY2:
> ++ viv_fmt = GL_VIV_YUY2;
> ++ break;
> ++ case GST_VIDEO_FORMAT_UYVY:
> ++ viv_fmt = GL_VIV_UYVY;
> ++ break;
> ++ case GST_VIDEO_FORMAT_RGBA:
> ++ viv_fmt = GL_RGBA;
> ++ break;
> ++ case GST_VIDEO_FORMAT_BGRA:
> ++ viv_fmt = GL_BGRA_EXT;
> ++ break;
> ++ case GST_VIDEO_FORMAT_RGB16:
> ++ viv_fmt = GL_RGB565_OES;
> ++ break;
> ++ default:
> ++ GST_ERROR ("Not supported format %d for viv direct texture upload.", fmt);
> ++ viv_fmt = GL_NONE;
> ++ return FALSE;
> ++ }
> ++
> ++ GstVivDirectTexture viv_tex = {tex_id, width, height, viv_fmt, memblk->vaddr, memblk->paddr, FALSE};
> ++ gst_gl_context_thread_add (context, (GstGLContextThreadFunc) _do_viv_direct_tex_bind_mem, &viv_tex);
> ++
> ++ return viv_tex.ret;
> ++}
> ++
> +diff --git a/gst-libs/gst/gl/gstglvivdirecttexture.h b/gst-libs/gst/gl/gstglvivdirecttexture.h
> +new file mode 100644
> +index 0000000..fa88e1a
> +--- /dev/null
> ++++ b/gst-libs/gst/gl/gstglvivdirecttexture.h
> +@@ -0,0 +1,35 @@
> ++/*
> ++ * GStreamer
> ++ * Copyright (c) 2015, Freescale Semiconductor, Inc.
> ++ *
> ++ * This library is free software; you can redistribute it and/or
> ++ * modify it under the terms of the GNU Library General Public
> ++ * License as published by the Free Software Foundation; either
> ++ * version 2 of the License, or (at your option) any later version.
> ++ *
> ++ * This library is distributed in the hope that it will be useful,
> ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
> ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> ++ * Library General Public License for more details.
> ++ *
> ++ * You should have received a copy of the GNU Library General Public
> ++ * License along with this library; if not, write to the
> ++ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
> ++ * Boston, MA 02110-1301, USA.
> ++ */
> ++
> ++#ifndef __GST_GL_VIVDIRECT_H__
> ++#define __GST_GL_VIVDIRECT_H__
> ++
> ++#include <gst/video/video.h>
> ++#include <gst/gl/gstgl_fwd.h>
> ++
> ++#define GST_GL_DIRECTVIV_FORMAT "{RGBA, I420, YV12, NV12, NV21, YUY2, UYVY, BGRA, RGB16}"
> ++G_BEGIN_DECLS
> ++
> ++gboolean gst_is_physical_buffer (GstBuffer *buffer);
> ++gboolean gst_gl_viv_direct_bind_gstbuffer (GstGLContext * context, guint tex_id, GstVideoInfo * info, GstBuffer * buffer);
> ++
> ++G_END_DECLS
> ++
> ++#endif /* __GST_GL_VIVDIRECT_H__ */
> +--
> +1.7.9.5
> +
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> index 7e6f3ff..0728009 100644
> --- a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> @@ -10,10 +10,13 @@ PACKAGECONFIG_GL_mx6sl = "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', \
> base_contains('DISTRO_FEATURES', 'x11', \
> 'opengl', '', d), '', d)}"
>
> -SRC_URI_append_mx6 = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
> -SRC_URI_append_mx6ul = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
> -SRC_URI_append_mx7 = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
> +IMX_PATCHES = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch \
> + file://0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch \
> +"
>
> +SRC_URI_append_mx6 = "${IMX_PATCHES}"
> +SRC_URI_append_mx6ul = "${IMX_PATCHES}"
> +SRC_URI_append_mx7 = "${IMX_PATCHES}"
>
> PACKAGE_ARCH_mx6 = "${MACHINE_SOCARCH}"
> PACKAGE_ARCH_mx6ul = "${MACHINE_SOCARCH}"
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance
2016-03-29 7:32 ` Carlos Rafael Giani
@ 2016-03-31 8:03 ` Yuqing Zhu
0 siblings, 0 replies; 8+ messages in thread
From: Yuqing Zhu @ 2016-03-31 8:03 UTC (permalink / raw)
To: Carlos Rafael Giani, Yuqing Zhu, meta-freescale
Hi Carlos,
Thanks for your good comments, and my answers are as followed.
B.R.
Yuqing Zhu
-----Original Message-----
From: Carlos Rafael Giani [mailto:dv@pseudoterminal.org]
Sent: Tuesday, March 29, 2016 3:33 PM
To: Yuqing Zhu <b54851@freescale.com>; meta-freescale@yoctoproject.org
Subject: Re: [meta-freescale] [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance
Generally this patch seems OK. There are some places that need clarification.
@@ -231,6 +233,18 @@ gst_gl_upload_element_prepare_output_buffer
(GstBaseTransform * bt,
return GST_FLOW_ERROR;
}
+ /* check if the output buffer's meta format equals to the caps format */
+ GstVideoMeta *vmeta = gst_buffer_get_video_meta(*outbuf);
+ gst_video_info_from_caps (&info, upload->out_caps);
+ if(vmeta->format != GST_VIDEO_INFO_FORMAT (&info))
+ {
+ caps = gst_caps_copy(upload->out_caps);
+ gst_caps_set_simple (caps, "format", G_TYPE_STRING,
+ gst_video_format_to_string (vmeta->format), NULL);
+ gst_caps_replace (&upload->out_caps, caps);
+ gst_base_transform_update_src_caps (upload, caps);
+ }
+
This is to ensure that the output format always corresponds to the
actual video frame's format? This is because the direct textures can
handle pixel format conversions automatically, right? But what if the
meta contains a format that is not supported by the direct texture?
(Y42B for example, which is not uncommon with USB webcams)
[Yuqing Zhu] Yes, direct textures can handle pixel format conversions automatically and only can output RGBA format. This output meta only will contain RGBA format. It is because there is a negotiate issue which the output caps will not be RGBA.
@@ -998,7 +1110,7 @@ gst_gl_upload_transform_caps (GstGLContext *
context, GstPadDirection direction,
if (filter) {
- result = gst_caps_intersect_full (filter, tmp,
GST_CAPS_INTERSECT_FIRST);
+ result = gst_caps_intersect_full (tmp, filter,
GST_CAPS_INTERSECT_FIRST);
gst_caps_unref (tmp);
} else {
result = tmp;
Why is this done?
[Yuqing Zhu] Caps intersect result will keep the first caps formats order. We want to try hard to let the element select the caps format from the "tmp" caps's order.
@@ -896,6 +896,11 @@ gst_glimage_sink_query (GstBaseSink * bsink,
GstQuery * query)
res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
break;
}
+ case GST_QUERY_ALLOCATION:
+ {
+ res = gst_glimage_sink_propose_allocation(bsink, query);
+ break;
+ }
default:
res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
break;
This one is interesting. Without this, there will be a memory copy?
[Yuqing Zhu] The default GST_QUERY_ALLOCATION will do a deep memory copy which will cause large loading latency.
Seems counter intuitive for glimagesink to do this.
Also, I think a similar patch was being used in a separate project. I
will look it up. Perhaps these can be merged.
On 2016-02-02 15:01, Yuqing Zhu wrote:
> 1. Add a physical buffer uploader in glupload plugin and using viv direct
> texture to bind physical continious buffer with texture to avoid memory
> copy from videobuffer to texture to gain good performance.
>
> 2. Reduce glimagesink load latency by override glimagesink ALLOCATION query to
> avoid memory copy.
>
> Signed-off-by: Yuqing Zhu <b54851@freescale.com>
> ---
> ...imagesink-to-improve-playback-performance.patch | 454 +++++++++++++++++++++
> .../gstreamer/gstreamer1.0-plugins-bad_%.bbappend | 9 +-
> 2 files changed, 460 insertions(+), 3 deletions(-)
> create mode 100755 recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
>
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
> new file mode 100755
> index 0000000..2739355
> --- /dev/null
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad/0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch
> @@ -0,0 +1,454 @@
> +From 587db892fdaac18780c8024e5a976ea182f36ff4 Mon Sep 17 00:00:00 2001
> +From: Haihua Hu <b55597@freescale.com>
> +Date: Tue, 13 Oct 2015 09:33:54 +0800
> +Subject: [PATCH 1/2] [glplugin] Add directviv to glimagesink to improve
> + playback performance
> +MIME-Version: 1.0
> +Content-Type: text/plain; charset=UTF-8
> +Content-Transfer-Encoding: 8bit
> +
> +1.Add a physical buffer uploader in glupload plugin and using viv direct
> + texture to bind physical continious buffer with texture to avoid memory
> + copy from videobuffer to texture to gain good performance.
> +2.Reduce glimagesink load latency by override glimagesink ALLOCATION query to
> + avoid memory copy.
> +
> +Upstream-Status: Inappropriate [i.MX specific]
> +
> +Signed-off-by: Haihua Hu <b55597@freescale.com>
> +---
> + ext/gl/gstglimagesink.c | 5 ++
> + ext/gl/gstgluploadelement.c | 14 +++
> + gst-libs/gst/gl/Makefile.am | 2 +
> + gst-libs/gst/gl/gstglupload.c | 118 ++++++++++++++++++++++++-
> + gst-libs/gst/gl/gstglvivdirecttexture.c | 143 +++++++++++++++++++++++++++++++
> + gst-libs/gst/gl/gstglvivdirecttexture.h | 35 ++++++++
> + 6 files changed, 314 insertions(+), 3 deletions(-)
> + create mode 100644 gst-libs/gst/gl/gstglvivdirecttexture.c
> + create mode 100644 gst-libs/gst/gl/gstglvivdirecttexture.h
> +
> +diff --git a/ext/gl/gstglimagesink.c b/ext/gl/gstglimagesink.c
> +index f629df7..1e5dc05 100644
> +--- a/ext/gl/gstglimagesink.c
> ++++ b/ext/gl/gstglimagesink.c
> +@@ -896,6 +896,11 @@ gst_glimage_sink_query (GstBaseSink * bsink, GstQuery * query)
> + res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
> + break;
> + }
> ++ case GST_QUERY_ALLOCATION:
> ++ {
> ++ res = gst_glimage_sink_propose_allocation(bsink, query);
> ++ break;
> ++ }
> + default:
> + res = GST_BASE_SINK_CLASS (parent_class)->query (bsink, query);
> + break;
> +diff --git a/ext/gl/gstgluploadelement.c b/ext/gl/gstgluploadelement.c
> +index 83ff24c..e74da3a 100644
> +--- a/ext/gl/gstgluploadelement.c
> ++++ b/ext/gl/gstgluploadelement.c
> +@@ -208,6 +208,8 @@ gst_gl_upload_element_prepare_output_buffer (GstBaseTransform * bt,
> + GstGLUploadElement *upload = GST_GL_UPLOAD_ELEMENT (bt);
> + GstGLUploadReturn ret;
> + GstBaseTransformClass *bclass;
> ++ GstVideoInfo info;
> ++ GstCaps *caps;
> +
> + bclass = GST_BASE_TRANSFORM_GET_CLASS (bt);
> +
> +@@ -231,6 +233,18 @@ gst_gl_upload_element_prepare_output_buffer (GstBaseTransform * bt,
> + return GST_FLOW_ERROR;
> + }
> +
> ++ /* check if the output buffer's meta format equals to the caps format */
> ++ GstVideoMeta *vmeta = gst_buffer_get_video_meta(*outbuf);
> ++ gst_video_info_from_caps (&info, upload->out_caps);
> ++ if(vmeta->format != GST_VIDEO_INFO_FORMAT (&info))
> ++ {
> ++ caps = gst_caps_copy(upload->out_caps);
> ++ gst_caps_set_simple (caps, "format", G_TYPE_STRING,
> ++ gst_video_format_to_string (vmeta->format), NULL);
> ++ gst_caps_replace (&upload->out_caps, caps);
> ++ gst_base_transform_update_src_caps (upload, caps);
> ++ }
> ++
> + /* basetransform doesn't unref if they're the same */
> + if (buffer == *outbuf)
> + gst_buffer_unref (*outbuf);
> +diff --git a/gst-libs/gst/gl/Makefile.am b/gst-libs/gst/gl/Makefile.am
> +index 36f3bf7..402e37a 100644
> +--- a/gst-libs/gst/gl/Makefile.am
> ++++ b/gst-libs/gst/gl/Makefile.am
> +@@ -30,6 +30,7 @@ libgstgl_@GST_API_VERSION@_la_SOURCES = \
> + gstglsyncmeta.c \
> + gstglviewconvert.c \
> + gstgloverlaycompositor.c \
> ++ gstglvivdirecttexture.c \
> + utils/opengl_versions.h
> +
> + libgstgl_@GST_API_VERSION@includedir = $(includedir)/gstreamer-@GST_API_VERSION@/gst/gl
> +@@ -56,6 +57,7 @@ libgstgl_@GST_API_VERSION@include_HEADERS = \
> + gstglviewconvert.h \
> + gstgloverlaycompositor.h \
> + gstgl_fwd.h \
> ++ gstglvivdirecttexture.h \
> + gl.h
> +
> + libgstgl_@GST_API_VERSION@_la_LIBADD = \
> +diff --git a/gst-libs/gst/gl/gstglupload.c b/gst-libs/gst/gl/gstglupload.c
> +index 0addca1..acaa329 100644
> +--- a/gst-libs/gst/gl/gstglupload.c
> ++++ b/gst-libs/gst/gl/gstglupload.c
> +@@ -23,6 +23,7 @@
> + #endif
> +
> + #include <stdio.h>
> ++#include <gst/gl/gstglvivdirecttexture.h>
> +
> + #include "gl.h"
> + #include "gstglupload.h"
> +@@ -47,7 +48,7 @@
> + #define USING_GLES2(context) (gst_gl_context_check_gl_version (context, GST_GL_API_GLES2, 2, 0))
> + #define USING_GLES3(context) (gst_gl_context_check_gl_version (context, GST_GL_API_GLES2, 3, 0))
> +
> +-GST_DEBUG_CATEGORY_STATIC (gst_gl_upload_debug);
> ++GST_DEBUG_CATEGORY (gst_gl_upload_debug);
> + #define GST_CAT_DEFAULT gst_gl_upload_debug
> +
> + #define DEBUG_INIT \
> +@@ -698,6 +699,114 @@ static const UploadMethod _upload_meta_upload = {
> + &_upload_meta_upload_free
> + };
> +
> ++struct PhyBufferUpload
> ++{
> ++ GstGLUpload *upload;
> ++};
> ++
> ++static gpointer
> ++_physical_buffer_upload_new(GstGLUpload *upload)
> ++{
> ++ struct PhyBufferUpload *phybuffer = g_new0 (struct PhyBufferUpload, 1);
> ++
> ++ phybuffer->upload = upload;
> ++
> ++ return phybuffer;
> ++}
> ++
> ++static GstCaps *
> ++_physical_buffer_upload_transform_caps(GstGLContext *context,
> ++ GstPadDirection direction, GstCaps *caps)
> ++{
> ++ GstCaps *ret;
> ++
> ++ if (direction == GST_PAD_SINK) {
> ++ ret = _set_caps_features (caps, GST_CAPS_FEATURE_MEMORY_GL_MEMORY);
> ++ gst_caps_set_simple (ret, "format", G_TYPE_STRING, "RGBA", NULL);
> ++ } else {
> ++ ret = gst_caps_from_string (GST_VIDEO_CAPS_MAKE_WITH_FEATURES
> ++ (GST_CAPS_FEATURE_MEMORY_SYSTEM_MEMORY,GST_GL_DIRECTVIV_FORMAT));
> ++ }
> ++
> ++ return ret;
> ++}
> ++
> ++static gboolean
> ++_physical_buffer_upload_accept(gpointer impl, GstBuffer *buffer,
> ++ GstCaps *in_caps, GstCaps *out_caps)
> ++{
> ++ GstCapsFeatures *features;
> ++
> ++ features = gst_caps_get_features (out_caps, 0);
> ++ if (!gst_caps_features_contains (features, GST_CAPS_FEATURE_MEMORY_GL_MEMORY))
> ++ return FALSE;
> ++
> ++ return gst_is_physical_buffer(buffer);
> ++}
> ++
> ++static void
> ++_physical_buffer_upload_propose_allocation(gpointer impl, GstQuery *decide_query,
> ++ GstQuery *query)
> ++{
> ++ gst_query_add_allocation_meta (query, GST_VIDEO_META_API_TYPE, 0);
> ++}
> ++
> ++static GstGLUploadReturn
> ++_physical_buffer_upload_perform(gpointer impl, GstBuffer *buffer, GstBuffer **outbuf)
> ++{
> ++ struct PhyBufferUpload *phyBuffer = impl;
> ++ GstVideoInfo *info;
> ++ gint n_mem;
> ++
> ++ info = &phyBuffer->upload->priv->out_info;
> ++ n_mem = GST_VIDEO_INFO_N_PLANES (info);
> ++ GST_LOG_OBJECT (phyBuffer->upload, "Attempting viv direct upload");
> ++ *outbuf = gst_buffer_new ();
> ++ gst_gl_memory_setup_buffer (phyBuffer->upload->context,
> ++ NULL, info, NULL, *outbuf);
> ++
> ++ GstGLMemory *out_gl_mem =
> ++ (GstGLMemory *) gst_buffer_peek_memory (*outbuf, 0);
> ++
> ++ gst_gl_viv_direct_bind_gstbuffer(phyBuffer->upload->context, out_gl_mem->tex_id,
> ++ &phyBuffer->upload->priv->in_info, buffer);
> ++
> ++ gst_buffer_add_video_meta_full (*outbuf, 0,
> ++ GST_VIDEO_INFO_FORMAT (info), GST_VIDEO_INFO_WIDTH (info),
> ++ GST_VIDEO_INFO_HEIGHT (info), n_mem, info->offset, info->stride);
> ++
> ++ return GST_GL_UPLOAD_DONE;
> ++}
> ++
> ++static void
> ++_physical_buffer_upload_release(gpointer impl, GstBuffer *buffer)
> ++{
> ++}
> ++
> ++static void
> ++_physical_buffer_upload_free(gpointer impl)
> ++{
> ++ struct PhyBufferUpload *phyBuffer = impl;
> ++
> ++ g_free(phyBuffer);
> ++}
> ++
> ++static GstStaticCaps _physical_buffer_upload_caps =
> ++GST_STATIC_CAPS (GST_VIDEO_CAPS_MAKE (GST_GL_DIRECTVIV_FORMAT));
> ++
> ++static const UploadMethod _physical_buffer_upload = {
> ++ "Physical buffer",
> ++ 0,
> ++ &_physical_buffer_upload_caps,
> ++ &_physical_buffer_upload_new,
> ++ &_physical_buffer_upload_transform_caps,
> ++ &_physical_buffer_upload_accept,
> ++ &_physical_buffer_upload_propose_allocation,
> ++ &_physical_buffer_upload_perform,
> ++ &_physical_buffer_upload_release,
> ++ &_physical_buffer_upload_free
> ++};
> ++
> + struct RawUploadFrame
> + {
> + gint ref_count;
> +@@ -813,6 +922,7 @@ _raw_data_upload_perform (gpointer impl, GstBuffer * buffer,
> + struct RawUpload *raw = impl;
> + int i;
> + GstVideoInfo *in_info = &raw->upload->priv->in_info;
> ++
> + guint max_planes = GST_VIDEO_INFO_N_PLANES (in_info);
> +
> + /* Support stereo views for separated multiview mode */
> +@@ -870,7 +980,9 @@ static const UploadMethod *upload_methods[] = { &_gl_memory_upload,
> + #if GST_GL_HAVE_PLATFORM_EGL
> + &_egl_image_upload,
> + #endif
> +- &_upload_meta_upload, &_raw_data_upload
> ++ &_upload_meta_upload,
> ++ &_physical_buffer_upload,
> ++ &_raw_data_upload
> + };
> +
> + static GMutex upload_global_lock;
> +@@ -998,7 +1110,7 @@ gst_gl_upload_transform_caps (GstGLContext * context, GstPadDirection direction,
> +
> +
> + if (filter) {
> +- result = gst_caps_intersect_full (filter, tmp, GST_CAPS_INTERSECT_FIRST);
> ++ result = gst_caps_intersect_full (tmp, filter, GST_CAPS_INTERSECT_FIRST);
> + gst_caps_unref (tmp);
> + } else {
> + result = tmp;
> +diff --git a/gst-libs/gst/gl/gstglvivdirecttexture.c b/gst-libs/gst/gl/gstglvivdirecttexture.c
> +new file mode 100644
> +index 0000000..c19b617
> +--- /dev/null
> ++++ b/gst-libs/gst/gl/gstglvivdirecttexture.c
> +@@ -0,0 +1,143 @@
> ++/*
> ++ * GStreamer
> ++ * Copyright (c) 2015, Freescale Semiconductor, Inc.
> ++ *
> ++ * This library is free software; you can redistribute it and/or
> ++ * modify it under the terms of the GNU Library General Public
> ++ * License as published by the Free Software Foundation; either
> ++ * version 2 of the License, or (at your option) any later version.
> ++ *
> ++ * This library is distributed in the hope that it will be useful,
> ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
> ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> ++ * Library General Public License for more details.
> ++ *
> ++ * You should have received a copy of the GNU Library General Public
> ++ * License along with this library; if not, write to the
> ++ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
> ++ * Boston, MA 02110-1301, USA.
> ++ */
> ++
> ++#ifdef HAVE_CONFIG_H
> ++#include "config.h"
> ++#endif
> ++
> ++#include "gl.h"
> ++
> ++GST_DEBUG_CATEGORY_EXTERN (gst_gl_upload_debug);
> ++#define GST_CAT_DEFAULT gst_gl_upload_debug
> ++
> ++typedef struct {
> ++ guint tex_id;
> ++ guint w;
> ++ guint h;
> ++ guint fmt;
> ++ void *vaddr;
> ++ guint paddr;
> ++ gboolean ret;
> ++} GstVivDirectTexture;
> ++
> ++gboolean
> ++gst_is_physical_buffer (GstBuffer *buffer)
> ++{
> ++
> ++ GstMemory *mem;
> ++
> ++ mem = gst_buffer_peek_memory (buffer, 0);
> ++ if (!mem->allocator)
> ++ return FALSE;
> ++
> ++ return g_type_check_instance_is_a (mem->allocator, g_type_from_name("GstAllocatorPhyMem"));
> ++}
> ++
> ++static void
> ++_do_viv_direct_tex_bind_mem (GstGLContext * context, GstVivDirectTexture * viv_tex)
> ++{
> ++ GST_DEBUG ("viv direct upload, tex_id %d, fmt: %d, res: (%dx%d)", viv_tex->tex_id, viv_tex->fmt, viv_tex->w, viv_tex->h);
> ++ GST_DEBUG ("Physical memory buffer, vaddr: %p, paddr: %p", viv_tex->vaddr, viv_tex->paddr);
> ++
> ++ glBindTexture (GL_TEXTURE_2D, viv_tex->tex_id);
> ++ glTexDirectVIVMap (GL_TEXTURE_2D, viv_tex->w, viv_tex->h, viv_tex->fmt, &viv_tex->vaddr, &viv_tex->paddr);
> ++ glTexDirectInvalidateVIV (GL_TEXTURE_2D);
> ++ viv_tex->ret = TRUE;
> ++
> ++ return;
> ++}
> ++
> ++gboolean
> ++gst_gl_viv_direct_bind_gstbuffer (GstGLContext * context, guint tex_id, GstVideoInfo * info, GstBuffer * buffer)
> ++{
> ++ typedef struct {
> ++ guint8 *vaddr;
> ++ guint8 *paddr;
> ++ guint8 *caddr;
> ++ gsize size;
> ++ gpointer *user_data;
> ++ } PhyMemBlock;
> ++ //Note: structure PhyMemBlock is copied from gst1.0-fsl-plugin/libs/allocator/gstallocatorphymem.h
> ++
> ++ typedef struct {
> ++ GstMemory mem;
> ++ guint8 *vaddr;
> ++ guint8 *paddr;
> ++ PhyMemBlock block;
> ++ } GstMemoryPhy;
> ++ //Note: structure GstMemoryPhy is copied from gst1.0-fsl-plugin/libs/allocator/gstallocatorphymem.c
> ++
> ++ GstMemory *mem = gst_buffer_peek_memory (buffer, 0);
> ++ GstMemoryPhy *memphy = (GstMemoryPhy*) mem;
> ++ PhyMemBlock *memblk = &memphy->block;
> ++
> ++ GstVideoFormat fmt = GST_VIDEO_INFO_FORMAT (info);
> ++ gint width, height;
> ++ GstVideoMeta *vmeta = gst_buffer_get_video_meta (buffer);
> ++ if (vmeta && (fmt == GST_VIDEO_FORMAT_I420 || fmt == GST_VIDEO_FORMAT_NV12)) {
> ++ width = vmeta->stride[0];
> ++ height = vmeta->offset[1] / width;
> ++ }
> ++ else {
> ++ width = GST_VIDEO_INFO_WIDTH (info);
> ++ height = GST_VIDEO_INFO_HEIGHT (info);
> ++ }
> ++
> ++ guint viv_fmt;
> ++ switch (fmt) {
> ++ case GST_VIDEO_FORMAT_I420:
> ++ viv_fmt = GL_VIV_I420;
> ++ break;
> ++ case GST_VIDEO_FORMAT_YV12:
> ++ viv_fmt = GL_VIV_YV12;
> ++ break;
> ++ case GST_VIDEO_FORMAT_NV12:
> ++ viv_fmt = GL_VIV_NV12;
> ++ break;
> ++ case GST_VIDEO_FORMAT_NV21:
> ++ viv_fmt = GL_VIV_NV21;
> ++ break;
> ++ case GST_VIDEO_FORMAT_YUY2:
> ++ viv_fmt = GL_VIV_YUY2;
> ++ break;
> ++ case GST_VIDEO_FORMAT_UYVY:
> ++ viv_fmt = GL_VIV_UYVY;
> ++ break;
> ++ case GST_VIDEO_FORMAT_RGBA:
> ++ viv_fmt = GL_RGBA;
> ++ break;
> ++ case GST_VIDEO_FORMAT_BGRA:
> ++ viv_fmt = GL_BGRA_EXT;
> ++ break;
> ++ case GST_VIDEO_FORMAT_RGB16:
> ++ viv_fmt = GL_RGB565_OES;
> ++ break;
> ++ default:
> ++ GST_ERROR ("Not supported format %d for viv direct texture upload.", fmt);
> ++ viv_fmt = GL_NONE;
> ++ return FALSE;
> ++ }
> ++
> ++ GstVivDirectTexture viv_tex = {tex_id, width, height, viv_fmt, memblk->vaddr, memblk->paddr, FALSE};
> ++ gst_gl_context_thread_add (context, (GstGLContextThreadFunc) _do_viv_direct_tex_bind_mem, &viv_tex);
> ++
> ++ return viv_tex.ret;
> ++}
> ++
> +diff --git a/gst-libs/gst/gl/gstglvivdirecttexture.h b/gst-libs/gst/gl/gstglvivdirecttexture.h
> +new file mode 100644
> +index 0000000..fa88e1a
> +--- /dev/null
> ++++ b/gst-libs/gst/gl/gstglvivdirecttexture.h
> +@@ -0,0 +1,35 @@
> ++/*
> ++ * GStreamer
> ++ * Copyright (c) 2015, Freescale Semiconductor, Inc.
> ++ *
> ++ * This library is free software; you can redistribute it and/or
> ++ * modify it under the terms of the GNU Library General Public
> ++ * License as published by the Free Software Foundation; either
> ++ * version 2 of the License, or (at your option) any later version.
> ++ *
> ++ * This library is distributed in the hope that it will be useful,
> ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of
> ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> ++ * Library General Public License for more details.
> ++ *
> ++ * You should have received a copy of the GNU Library General Public
> ++ * License along with this library; if not, write to the
> ++ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
> ++ * Boston, MA 02110-1301, USA.
> ++ */
> ++
> ++#ifndef __GST_GL_VIVDIRECT_H__
> ++#define __GST_GL_VIVDIRECT_H__
> ++
> ++#include <gst/video/video.h>
> ++#include <gst/gl/gstgl_fwd.h>
> ++
> ++#define GST_GL_DIRECTVIV_FORMAT "{RGBA, I420, YV12, NV12, NV21, YUY2, UYVY, BGRA, RGB16}"
> ++G_BEGIN_DECLS
> ++
> ++gboolean gst_is_physical_buffer (GstBuffer *buffer);
> ++gboolean gst_gl_viv_direct_bind_gstbuffer (GstGLContext * context, guint tex_id, GstVideoInfo * info, GstBuffer * buffer);
> ++
> ++G_END_DECLS
> ++
> ++#endif /* __GST_GL_VIVDIRECT_H__ */
> +--
> +1.7.9.5
> +
> diff --git a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> index 7e6f3ff..0728009 100644
> --- a/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> +++ b/recipes-multimedia/gstreamer/gstreamer1.0-plugins-bad_%.bbappend
> @@ -10,10 +10,13 @@ PACKAGECONFIG_GL_mx6sl = "${@bb.utils.contains('DISTRO_FEATURES', 'opengl', \
> base_contains('DISTRO_FEATURES', 'x11', \
> 'opengl', '', d), '', d)}"
>
> -SRC_URI_append_mx6 = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
> -SRC_URI_append_mx6ul = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
> -SRC_URI_append_mx7 = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch"
> +IMX_PATCHES = " file://0001-PATCH-install-gstaggregator-and-gstvideoaggregator-h.patch \
> + file://0002-glplugin-Add-directviv-to-glimagesink-to-improve-playback-performance.patch \
> +"
>
> +SRC_URI_append_mx6 = "${IMX_PATCHES}"
> +SRC_URI_append_mx6ul = "${IMX_PATCHES}"
> +SRC_URI_append_mx7 = "${IMX_PATCHES}"
>
> PACKAGE_ARCH_mx6 = "${MACHINE_SOCARCH}"
> PACKAGE_ARCH_mx6ul = "${MACHINE_SOCARCH}"
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-03-31 8:03 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-02 14:01 [meta-fsl-arm][PATCH v2 0/3] Add patches for gstreamer1.0-plugins-bad 1.6 Yuqing Zhu
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 1/3] gstreamer1.0-plugins-bad: Add directviv to glimagesink to improve playback performance Yuqing Zhu
2016-03-29 7:32 ` Carlos Rafael Giani
2016-03-31 8:03 ` Yuqing Zhu
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 2/3] gstreamer1.0-plugins-bad: Support video crop for glimagesink Yuqing Zhu
2016-03-29 7:14 ` Carlos Rafael Giani
2016-02-02 14:01 ` [meta-fsl-arm][PATCH v2 3/3] gstreamer1.0-plugins-bad: Change wayland default resolution to 1024x768 Yuqing Zhu
2016-03-29 7:07 ` Carlos Rafael Giani
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.