[RFC][PATCH 1/2] drm_hwcomposer: Error out on YUV layer as it would fail for single planes

From: John Stultz <john.stultz@linaro.org>
To: dri-devel <dri-devel@lists.freedesktop.org>
Cc: Rob Herring <rob.herring@linaro.org>,
	Matt Szczesiak <matt.szczesiak@arm.com>,
	Dmitry Shmidt <dimitrysh@google.com>,
	Sean Paul <seanpaul@google.com>,
	Robert Foss <robert.foss@collabora.com>,
	Alexandru-Cosmin Gheorghe <Alexandru-Cosmin.Gheorghe@arm.com>,
	Liviu Dudau <Liviu.Dudau@arm.com>,
	Marissa Wall <marissaw@google.com>,
	David Hanna <david.hanna11@gmail.com>
Subject: [RFC][PATCH 1/2] drm_hwcomposer: Error out on YUV layer as it would fail for single planes
Date: Tue,  6 Mar 2018 15:19:11 -0800	[thread overview]
Message-ID: <1520378352-31260-1-git-send-email-john.stultz@linaro.org> (raw)

As suggested by Alexandru-Cosmin Gheorghe:

ConvertHALFormatToDrm logic would work only for 1 plane formats,
and probably gets rejected by drmModeAddFb2, but to save
debugging time  maybe it worth removing DRM_FORMAT_YVU420 from
ConvertHALFormatToDrm and checking it's return code.

So this patch tries to do this.

Cc: Marissa Wall <marissaw@google.com>
Cc: Sean Paul <seanpaul@google.com>
Cc: Dmitry Shmidt <dimitrysh@google.com>
Cc: Robert Foss <robert.foss@collabora.com>
Cc: Matt Szczesiak <matt.szczesiak@arm.com>
Cc: Liviu Dudau <Liviu.Dudau@arm.com>
Cc: David Hanna <david.hanna11@gmail.com>
Cc: Rob Herring <rob.herring@linaro.org>
Cc: Alexandru-Cosmin Gheorghe <Alexandru-Cosmin.Gheorghe@arm.com>
Signed-off-by: John Stultz <john.stultz@linaro.org>
---
 platformdrmgeneric.cpp | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/platformdrmgeneric.cpp b/platformdrmgeneric.cpp
index 741d42b..33f1ea0 100644
--- a/platformdrmgeneric.cpp
+++ b/platformdrmgeneric.cpp
@@ -76,8 +76,6 @@ uint32_t DrmGenericImporter::ConvertHalFormatToDrm(uint32_t hal_format) {
       return DRM_FORMAT_ABGR8888;
     case HAL_PIXEL_FORMAT_RGB_565:
       return DRM_FORMAT_BGR565;
-    case HAL_PIXEL_FORMAT_YV12:
-      return DRM_FORMAT_YVU420;
     default:
       ALOGE("Cannot convert hal format to drm format %u", hal_format);
       return -EINVAL;
@@ -88,10 +86,15 @@ EGLImageKHR DrmGenericImporter::ImportImage(EGLDisplay egl_display, buffer_handl
   gralloc_drm_handle_t *gr_handle = gralloc_drm_handle(handle);
   if (!gr_handle)
     return NULL;
+
+  EGLint fmt = ConvertHalFormatToDrm(gr_handle->format);
+  if (fmt < 0)
+	return NULL;
+
   EGLint attr[] = {
     EGL_WIDTH, gr_handle->width,
     EGL_HEIGHT, gr_handle->height,
-    EGL_LINUX_DRM_FOURCC_EXT, (EGLint)ConvertHalFormatToDrm(gr_handle->format),
+    EGL_LINUX_DRM_FOURCC_EXT, fmt,
     EGL_DMA_BUF_PLANE0_FD_EXT, gr_handle->prime_fd,
     EGL_DMA_BUF_PLANE0_OFFSET_EXT, 0,
     EGL_DMA_BUF_PLANE0_PITCH_EXT, gr_handle->stride,
@@ -112,10 +115,14 @@ int DrmGenericImporter::ImportBuffer(buffer_handle_t handle, hwc_drm_bo_t *bo) {
     return ret;
   }
 
+  uint32_t  fmt = ConvertHalFormatToDrm(gr_handle->format);
+  if (fmt < 0)
+        return fmt;
+
   memset(bo, 0, sizeof(hwc_drm_bo_t));
   bo->width = gr_handle->width;
   bo->height = gr_handle->height;
-  bo->format = ConvertHalFormatToDrm(gr_handle->format);
+  bo->format = fmt;
   bo->usage = gr_handle->usage;
   bo->pitches[0] = gr_handle->stride;
   bo->gem_handles[0] = gem_handle;
-- 
2.7.4

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
