[PATCH v2 00/54] R-Car VSP improvements for v4.7

[PATCH v2 00/54] R-Car VSP improvements for v4.7

[Laurent Pinchart]

Hello,

This patch series contains all the pending vsp1 driver improvements for v4.7.
In particular, it enables display list usage in non-DRM pipelines (24/54) and
adds support for multi-body display lists (48/54) and the R-Car Gen3 RPF alpha
multiplier (50/54), Z-order control (51/54) and global alpha control (52/54).

The other patches are cleanups, bug fixes and refactoring to support the five
features listed above.

The code is based on top of the "[PATCH v6 0/2] media: Add entity types" patch
series. For convenience I've pushed a branch that contains all the necessary
patches on top of the latest Linux media master branch to

        git://linuxtv.org/pinchartl/media.git vsp1/next

Note that while patch 51/54 and 52/54 enable support for Z-order control and
global alpha control in the vsp1 driver, enabling the features for userspace
requires additional patches for the rcar-du-drm driver. I have pushed a branch
that includes the rcar-du-drm changes and platform enablements to

        git://linuxtv.org/pinchartl/media.git drm/du/vsp1-kms/boards


Changes since v1:

- Fix a typo in the commit message of 03/54
- Base RPF alpha multiplier detection on generation number instead of feature
  flag
- Add patches 52/54 to 54/54

Laurent Pinchart (54):
  media: Add video processing entity functions
  v4l: subdev: Add pad config allocator and init
  v4l: subdev: Call pad init_cfg operation when opening subdevs
  v4l: vsp1: Fix vsp1_du_atomic_(begin|flush) declarations
  v4l: vsp1: drm: Include correct header file
  v4l: vsp1: video: Fix coding style
  v4l: vsp1: Set entities functions
  v4l: vsp1: VSPD instances have no LUT on Gen3
  v4l: vsp1: Use pipeline display list to decide how to write to modules
  v4l: vsp1: Always setup the display list
  v4l: vsp1: Simplify frame end processing
  v4l: vsp1: Split display list manager from display list
  v4l: vsp1: Store the display list manager in the WPF
  v4l: vsp1: bru: Don't program background color in control set handler
  v4l: vsp1: rwpf: Don't program alpha value in control set handler
  v4l: vsp1: sru: Don't program intensity in control set handler
  v4l: vsp1: Don't setup control handler when starting streaming
  v4l: vsp1: Enable display list support for the HS[IT], LUT, SRU and
    UDS
  v4l: vsp1: Don't configure RPF memory buffers before calculating
    offsets
  v4l: vsp1: Remove unneeded entity streaming flag
  v4l: vsp1: Document calling context of vsp1_pipeline_propagate_alpha()
  v4l: vsp1: Fix 80 characters per line violations
  v4l: vsp1: Add header display list support
  v4l: vsp1: Use display lists with the userspace API
  v4l: vsp1: Move subdev initialization code to vsp1_entity_init()
  v4l: vsp1: Consolidate entity ops in a struct vsp1_entity_operations
  v4l: vsp1: Fix BRU try compose rectangle storage
  v4l: vsp1: Add race condition FIXME comment
  v4l: vsp1: Implement and use the subdev pad::init_cfg configuration
  v4l: vsp1: Store active formats in a pad config structure
  v4l: vsp1: Store active selection rectangles in a pad config structure
  v4l: vsp1: Create a new configure operation to setup modules
  v4l: vsp1: Merge RPF and WPF pad ops structures
  v4l: vsp1: Use __vsp1_video_try_format to initialize format at init
    time
  v4l: vsp1: Pass display list explicitly to configure functions
  v4l: vsp1: Rename pipeline validate functions to pipeline build
  v4l: vsp1: Pass pipe pointer to entity configure functions
  v4l: vsp1: Store pipeline pointer in rwpf
  v4l: vsp1: video: Reorder functions
  v4l: vsp1: Allocate pipelines on demand
  v4l: vsp1: RPF entities can't be target nodes
  v4l: vsp1: Factorize get pad format code
  v4l: vsp1: Factorize media bus codes enumeration code
  v4l: vsp1: Factorize frame size enumeration code
  v4l: vsp1: Fix LUT format setting
  v4l: vsp1: dl: Make reg_count field unsigned
  v4l: vsp1: dl: Fix race conditions
  v4l: vsp1: dl: Add support for multi-body display lists
  v4l: vsp1: lut: Use display list fragments to fill LUT
  v4l: vsp1: Add support for the RPF alpha multiplier on Gen3
  v4l: vsp1: Add Z-order support for DRM pipeline
  v4l: vsp1: Add global alpha support for DRM pipeline
  v4l: vsp1: Fix V4L2_PIX_FMT_XRGB444 format definition
  v4l: vsp1: Update WPF and LIF maximum sizes for Gen3

 Documentation/DocBook/media/v4l/media-types.xml |  34 ++
 drivers/media/platform/vsp1/vsp1.h              |  14 +-
 drivers/media/platform/vsp1/vsp1_bru.c          | 360 +++++++--------
 drivers/media/platform/vsp1/vsp1_bru.h          |   3 +-
 drivers/media/platform/vsp1/vsp1_dl.c           | 568 ++++++++++++++++++------
 drivers/media/platform/vsp1/vsp1_dl.h           |  49 +-
 drivers/media/platform/vsp1/vsp1_drm.c          | 234 +++++-----
 drivers/media/platform/vsp1/vsp1_drm.h          |  27 +-
 drivers/media/platform/vsp1/vsp1_drv.c          |  34 +-
 drivers/media/platform/vsp1/vsp1_entity.c       | 289 ++++++++----
 drivers/media/platform/vsp1/vsp1_entity.h       |  63 ++-
 drivers/media/platform/vsp1/vsp1_hsit.c         | 130 ++----
 drivers/media/platform/vsp1/vsp1_lif.c          | 180 +++-----
 drivers/media/platform/vsp1/vsp1_lut.c          | 173 +++-----
 drivers/media/platform/vsp1/vsp1_lut.h          |   6 +-
 drivers/media/platform/vsp1/vsp1_pipe.c         |  71 +--
 drivers/media/platform/vsp1/vsp1_pipe.h         |  19 +-
 drivers/media/platform/vsp1/vsp1_regs.h         |  10 +
 drivers/media/platform/vsp1/vsp1_rpf.c          | 276 ++++++------
 drivers/media/platform/vsp1/vsp1_rwpf.c         | 171 ++++---
 drivers/media/platform/vsp1/vsp1_rwpf.h         |  64 +--
 drivers/media/platform/vsp1/vsp1_sru.c          | 215 ++++-----
 drivers/media/platform/vsp1/vsp1_sru.h          |   2 +
 drivers/media/platform/vsp1/vsp1_uds.c          | 224 ++++------
 drivers/media/platform/vsp1/vsp1_uds.h          |   3 +-
 drivers/media/platform/vsp1/vsp1_video.c        | 493 +++++++++++---------
 drivers/media/platform/vsp1/vsp1_video.h        |   2 -
 drivers/media/platform/vsp1/vsp1_wpf.c          | 280 ++++++------
 drivers/media/v4l2-core/v4l2-subdev.c           |  37 +-
 include/media/v4l2-subdev.h                     |   8 +
 include/media/vsp1.h                            |  23 +-
 include/uapi/linux/media.h                      |   8 +
 32 files changed, 2186 insertions(+), 1884 deletions(-)

-- 
Regards,

Laurent Pinchart

[PATCH v2 01/54] media: Add video processing entity functions

[Laurent Pinchart]

Add composer, format converter and scaler functions, as well as generic
video processing to be used when no other processing function is
applicable.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 Documentation/DocBook/media/v4l/media-types.xml | 34 +++++++++++++++++++++++++
 include/uapi/linux/media.h                      |  8 ++++++
 2 files changed, 42 insertions(+)

diff --git a/Documentation/DocBook/media/v4l/media-types.xml b/Documentation/DocBook/media/v4l/media-types.xml
index 5e3f20fdcf17..a6e171e80bce 100644
--- a/Documentation/DocBook/media/v4l/media-types.xml
+++ b/Documentation/DocBook/media/v4l/media-types.xml
@@ -121,6 +121,40 @@
 	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
 	    <entry>Audio Mixer Function Entity.</entry>
 	  </row>
+	  <row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
+	    <entry>Generic video processing, when no other processing function
+		   is applicable.
+	    </entry>
+	  <row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
+	    <entry>Video composer (blender). An entity capable of video
+		   composing must have at least two sink pads and one source
+		   pad, and composes input video frames onto output video
+		   frames. Composition can be performed using alpha blending,
+		   color keying, raster operations (ROP), stitching or any other
+		   mean.
+	    </entry>
+	  </row>
+	  </row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
+	    <entry>Video format converter. An entity capable of video format
+		   conversion must have at least one sink pad and one source
+		   pad, and convert the format of pixels received on its sink
+		   pad(s) to a different format output on its source pad(s).
+	    </entry>
+	  </row>
+	  <row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
+	    <entry>Video scaler. An entity capable of video scaling must have
+		   at least one sink pad and one source pad, and scaling the
+		   video frame(s) received on its sink pad(s) to a different
+		   resolution output on its source pad(s). The range of
+		   supported scaling ratios is entity-specific and can differ
+		   between the horizontal and vertical directions. In particular
+		   scaling can be supported in one direction only.
+	    </entry>
+	  </row>
 	</tbody>
       </tgroup>
     </table>
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index df59edee25d1..884ec1cae09d 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -95,6 +95,14 @@ struct media_device_info {
 #define MEDIA_ENT_F_AUDIO_MIXER		(MEDIA_ENT_F_BASE + 0x03003)
 
 /*
+ * Processing entities
+ */
+#define MEDIA_ENT_F_PROC_VIDEO_GENERIC		(MEDIA_ENT_F_BASE + 0x4001)
+#define MEDIA_ENT_F_PROC_VIDEO_COMPOSER		(MEDIA_ENT_F_BASE + 0x4002)
+#define MEDIA_ENT_F_PROC_VIDEO_CONVERTER	(MEDIA_ENT_F_BASE + 0x4003)
+#define MEDIA_ENT_F_PROC_VIDEO_SCALER		(MEDIA_ENT_F_BASE + 0x4004)
+
+/*
  * Connectors
  */
 /* It is a responsibility of the entity drivers to add connectors and links */
-- 
2.7.3

[PATCH v2 02/54] v4l: subdev: Add pad config allocator and init

[Laurent Pinchart]

From: Laurent Pinchart <laurent.pinchart@linaro.org>

Add a new subdev operation to initialize a subdev pad config array, and
a helper function to allocate and initialize the array. This can be used
by bridge drivers to implement try format based on subdev pad
operations.

Signed-off-by: Laurent Pinchart <laurent.pinchart@linaro.org>
Acked-by: Vaibhav Hiremath <vaibhav.hiremath@linaro.org>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 31 ++++++++++++++++++++++++++++++-
 include/media/v4l2-subdev.h           |  8 ++++++++
 2 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 0fa60801a428..d4007f8f58d1 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -35,7 +35,7 @@
 static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
 {
 #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
-	fh->pad = kzalloc(sizeof(*fh->pad) * sd->entity.num_pads, GFP_KERNEL);
+	fh->pad = v4l2_subdev_alloc_pad_config(sd);
 	if (fh->pad == NULL)
 		return -ENOMEM;
 #endif
@@ -569,6 +569,35 @@ int v4l2_subdev_link_validate(struct media_link *link)
 		sink, link, &source_fmt, &sink_fmt);
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
+
+struct v4l2_subdev_pad_config *
+v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd)
+{
+	struct v4l2_subdev_pad_config *cfg;
+	int ret;
+
+	if (!sd->entity.num_pads)
+		return NULL;
+
+	cfg = kcalloc(sd->entity.num_pads, sizeof(*cfg), GFP_KERNEL);
+	if (!cfg)
+		return NULL;
+
+	ret = v4l2_subdev_call(sd, pad, init_cfg, cfg);
+	if (ret < 0 && ret != -ENOIOCTLCMD) {
+		kfree(cfg);
+		return NULL;
+	}
+
+	return cfg;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_alloc_pad_config);
+
+void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg)
+{
+	kfree(cfg);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_free_pad_config);
 #endif /* CONFIG_MEDIA_CONTROLLER */
 
 void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 11e2dfec0198..32fc7a4beb5e 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -572,6 +572,7 @@ struct v4l2_subdev_pad_config {
 /**
  * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
  *
+ * @init_cfg: initialize the pad config to default values
  * @enum_mbus_code: callback for VIDIOC_SUBDEV_ENUM_MBUS_CODE ioctl handler
  *		    code.
  * @enum_frame_size: callback for VIDIOC_SUBDEV_ENUM_FRAME_SIZE ioctl handler
@@ -607,6 +608,8 @@ struct v4l2_subdev_pad_config {
  *                  may be adjusted by the subdev driver to device capabilities.
  */
 struct v4l2_subdev_pad_ops {
+	int (*init_cfg)(struct v4l2_subdev *sd,
+			struct v4l2_subdev_pad_config *cfg);
 	int (*enum_mbus_code)(struct v4l2_subdev *sd,
 			      struct v4l2_subdev_pad_config *cfg,
 			      struct v4l2_subdev_mbus_code_enum *code);
@@ -801,7 +804,12 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
 				      struct v4l2_subdev_format *source_fmt,
 				      struct v4l2_subdev_format *sink_fmt);
 int v4l2_subdev_link_validate(struct media_link *link);
+
+struct v4l2_subdev_pad_config *
+v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd);
+void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg);
 #endif /* CONFIG_MEDIA_CONTROLLER */
+
 void v4l2_subdev_init(struct v4l2_subdev *sd,
 		      const struct v4l2_subdev_ops *ops);
 
-- 
2.7.3

[PATCH v2 03/54] v4l: subdev: Call pad init_cfg operation when opening subdevs

[Laurent Pinchart]

The subdev core code currently rely on the subdev open handler to
initialize the file handle's pad configuration, even though subdevs now
have a pad operation dedicated for that purpose.

As a first step towards migration to init_cfg, call the operation
operation in the subdev core open implementation. Subdevs that haven't
been moved to init_cfg yet will just continue implementing pad config
initialization in their open handler.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index d4007f8f58d1..1fa6b713ee19 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -83,6 +83,12 @@ static int subdev_open(struct file *file)
 	}
 #endif
 
+#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
+	ret = v4l2_subdev_call(sd, pad, init_cfg, subdev_fh->pad);
+	if (ret < 0 && ret != -ENOIOCTLCMD)
+		goto err;
+#endif
+
 	if (sd->internal_ops && sd->internal_ops->open) {
 		ret = sd->internal_ops->open(sd, subdev_fh);
 		if (ret < 0)
-- 
2.7.3

[PATCH v2 04/54] v4l: vsp1: Fix vsp1_du_atomic_(begin|flush) declarations

[Laurent Pinchart]

The functions are void, make the declaration match the definition.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 include/media/vsp1.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/media/vsp1.h b/include/media/vsp1.h
index cc541753896f..d01f7cb8f691 100644
--- a/include/media/vsp1.h
+++ b/include/media/vsp1.h
@@ -23,11 +23,11 @@ int vsp1_du_init(struct device *dev);
 int vsp1_du_setup_lif(struct device *dev, unsigned int width,
 		      unsigned int height);
 
-int vsp1_du_atomic_begin(struct device *dev);
+void vsp1_du_atomic_begin(struct device *dev);
 int vsp1_du_atomic_update(struct device *dev, unsigned int rpf, u32 pixelformat,
 			  unsigned int pitch, dma_addr_t mem[2],
 			  const struct v4l2_rect *src,
 			  const struct v4l2_rect *dst);
-int vsp1_du_atomic_flush(struct device *dev);
+void vsp1_du_atomic_flush(struct device *dev);
 
 #endif /* __MEDIA_VSP1_H__ */
-- 
2.7.3

[PATCH v2 05/54] v4l: vsp1: drm: Include correct header file

[Laurent Pinchart]

The VSP1 DRM API is declared in <media/vsp1.h>, not <linux/vsp1.h>. Fix
it. This also reverts commit 18922936dc28 ("[media] vsp1_drm.h: add
missing prototypes") that added the same declarations in a different
header file.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_drm.c |  2 +-
 drivers/media/platform/vsp1/vsp1_drm.h | 11 -----------
 2 files changed, 1 insertion(+), 12 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 021fe5778cd1..8cf7c19f4344 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -13,10 +13,10 @@
 
 #include <linux/device.h>
 #include <linux/slab.h>
-#include <linux/vsp1.h>
 
 #include <media/media-entity.h>
 #include <media/v4l2-subdev.h>
+#include <media/vsp1.h>
 
 #include "vsp1.h"
 #include "vsp1_bru.h"
diff --git a/drivers/media/platform/vsp1/vsp1_drm.h b/drivers/media/platform/vsp1/vsp1_drm.h
index f68056838319..7704038c3add 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.h
+++ b/drivers/media/platform/vsp1/vsp1_drm.h
@@ -35,15 +35,4 @@ int vsp1_drm_init(struct vsp1_device *vsp1);
 void vsp1_drm_cleanup(struct vsp1_device *vsp1);
 int vsp1_drm_create_links(struct vsp1_device *vsp1);
 
-int vsp1_du_init(struct device *dev);
-int vsp1_du_setup_lif(struct device *dev, unsigned int width,
-		      unsigned int height);
-void vsp1_du_atomic_begin(struct device *dev);
-int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
-			  u32 pixelformat, unsigned int pitch,
-			  dma_addr_t mem[2], const struct v4l2_rect *src,
-			  const struct v4l2_rect *dst);
-void vsp1_du_atomic_flush(struct device *dev);
-
-
 #endif /* __VSP1_DRM_H__ */
-- 
2.7.3

[PATCH v2 06/54] v4l: vsp1: video: Fix coding style

[Laurent Pinchart]

Commit 54b5a749b4f3 ("[media] v4l: vsp1: Use media entity enumeration
interface") wasn't aligned with the driver coding style. Fix it by
renaming the rval variable to ret.

Furthermore shorten lines by accessing the media_device instance in a
more straightforward fashion.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_video.c | 23 +++++++++++------------
 1 file changed, 11 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 72cc7d3729f8..b97bbdb1a256 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -175,31 +175,30 @@ static int vsp1_video_pipeline_validate_branch(struct vsp1_pipeline *pipe,
 					       struct vsp1_rwpf *input,
 					       struct vsp1_rwpf *output)
 {
-	struct vsp1_entity *entity;
 	struct media_entity_enum ent_enum;
+	struct vsp1_entity *entity;
 	struct media_pad *pad;
-	int rval;
 	bool bru_found = false;
+	int ret;
 
 	input->location.left = 0;
 	input->location.top = 0;
 
-	rval = media_entity_enum_init(
-		&ent_enum, input->entity.pads[RWPF_PAD_SOURCE].graph_obj.mdev);
-	if (rval)
-		return rval;
+	ret = media_entity_enum_init(&ent_enum, &input->entity.vsp1->media_dev);
+	if (ret < 0)
+		return ret;
 
 	pad = media_entity_remote_pad(&input->entity.pads[RWPF_PAD_SOURCE]);
 
 	while (1) {
 		if (pad == NULL) {
-			rval = -EPIPE;
+			ret = -EPIPE;
 			goto out;
 		}
 
 		/* We've reached a video node, that shouldn't have happened. */
 		if (!is_media_entity_v4l2_subdev(pad->entity)) {
-			rval = -EPIPE;
+			ret = -EPIPE;
 			goto out;
 		}
 
@@ -229,14 +228,14 @@ static int vsp1_video_pipeline_validate_branch(struct vsp1_pipeline *pipe,
 		/* Ensure the branch has no loop. */
 		if (media_entity_enum_test_and_set(&ent_enum,
 						   &entity->subdev.entity)) {
-			rval = -EPIPE;
+			ret = -EPIPE;
 			goto out;
 		}
 
 		/* UDS can't be chained. */
 		if (entity->type == VSP1_ENTITY_UDS) {
 			if (pipe->uds) {
-				rval = -EPIPE;
+				ret = -EPIPE;
 				goto out;
 			}
 
@@ -256,12 +255,12 @@ static int vsp1_video_pipeline_validate_branch(struct vsp1_pipeline *pipe,
 
 	/* The last entity must be the output WPF. */
 	if (entity != &output->entity)
-		rval = -EPIPE;
+		ret = -EPIPE;
 
 out:
 	media_entity_enum_cleanup(&ent_enum);
 
-	return rval;
+	return ret;
 }
 
 static int vsp1_video_pipeline_validate(struct vsp1_pipeline *pipe,
-- 
2.7.3

[PATCH v2 07/54] v4l: vsp1: Set entities functions

[Laurent Pinchart]

Initialize the function field of all subdev entities instantiated by the
driver. This gets rids of multiple warnings printed by the media
controller core.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c  | 2 ++
 drivers/media/platform/vsp1/vsp1_hsit.c | 2 ++
 drivers/media/platform/vsp1/vsp1_lif.c  | 2 ++
 drivers/media/platform/vsp1/vsp1_lut.c  | 2 ++
 drivers/media/platform/vsp1/vsp1_rpf.c  | 2 ++
 drivers/media/platform/vsp1/vsp1_sru.c  | 2 ++
 drivers/media/platform/vsp1/vsp1_uds.c  | 2 ++
 drivers/media/platform/vsp1/vsp1_wpf.c  | 2 ++
 8 files changed, 16 insertions(+)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index cb0dbc15ddad..565c8b2edf19 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -424,7 +424,9 @@ struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1)
 	subdev = &bru->entity.subdev;
 	v4l2_subdev_init(subdev, &bru_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_COMPOSER;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s bru",
 		 dev_name(vsp1->dev));
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index c1087cff31a0..ce42ce2e4847 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -203,7 +203,9 @@ struct vsp1_hsit *vsp1_hsit_create(struct vsp1_device *vsp1, bool inverse)
 	subdev = &hsit->entity.subdev;
 	v4l2_subdev_init(subdev, &hsit_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_CONVERTER;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s %s",
 		 dev_name(vsp1->dev), inverse ? "hsi" : "hst");
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 433853ce8dbf..56054fddb675 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -223,7 +223,9 @@ struct vsp1_lif *vsp1_lif_create(struct vsp1_device *vsp1)
 	subdev = &lif->entity.subdev;
 	v4l2_subdev_init(subdev, &lif_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_GENERIC;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s lif",
 		 dev_name(vsp1->dev));
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 4b89095e7b5f..f0cd4f79fbff 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -237,7 +237,9 @@ struct vsp1_lut *vsp1_lut_create(struct vsp1_device *vsp1)
 	subdev = &lut->entity.subdev;
 	v4l2_subdev_init(subdev, &lut_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_GENERIC;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s lut",
 		 dev_name(vsp1->dev));
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 5bc1d1574a43..7853e0f1d526 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -245,7 +245,9 @@ struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index)
 	subdev = &rpf->entity.subdev;
 	v4l2_subdev_init(subdev, &rpf_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_CONVERTER;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s rpf.%u",
 		 dev_name(vsp1->dev), index);
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index cc09efbfb24f..149ee1cd0b5a 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -363,7 +363,9 @@ struct vsp1_sru *vsp1_sru_create(struct vsp1_device *vsp1)
 	subdev = &sru->entity.subdev;
 	v4l2_subdev_init(subdev, &sru_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s sru",
 		 dev_name(vsp1->dev));
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index bba67770cf95..b1881a0a314f 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -338,7 +338,9 @@ struct vsp1_uds *vsp1_uds_create(struct vsp1_device *vsp1, unsigned int index)
 	subdev = &uds->entity.subdev;
 	v4l2_subdev_init(subdev, &uds_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s uds.%u",
 		 dev_name(vsp1->dev), index);
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index c78d4af50fcf..d2735f09d1da 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -244,7 +244,9 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	subdev = &wpf->entity.subdev;
 	v4l2_subdev_init(subdev, &wpf_ops);
 
+	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_CONVERTER;
 	subdev->entity.ops = &vsp1->media_ops;
+
 	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	snprintf(subdev->name, sizeof(subdev->name), "%s wpf.%u",
 		 dev_name(vsp1->dev), index);
-- 
2.7.3

[PATCH v2 08/54] v4l: vsp1: VSPD instances have no LUT on Gen3

[Laurent Pinchart]

Remove the HAS_LUT flag in the corresponding device information entry.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index 25750a0e4631..da43e3f35610 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -623,7 +623,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPD_GEN3,
-		.features = VSP1_HAS_BRU | VSP1_HAS_LIF | VSP1_HAS_LUT,
+		.features = VSP1_HAS_BRU | VSP1_HAS_LIF,
 		.rpf_count = 5,
 		.wpf_count = 2,
 		.num_bru_inputs = 5,
-- 
2.7.3

[PATCH v2 09/54] v4l: vsp1: Use pipeline display list to decide how to write to modules

[Laurent Pinchart]

This allows getting rid of the vsp1_device::use_dl field.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1.h        | 12 ------------
 drivers/media/platform/vsp1/vsp1_dl.c     |  5 +----
 drivers/media/platform/vsp1/vsp1_dl.h     | 12 ++----------
 drivers/media/platform/vsp1/vsp1_drv.c    |  9 +++------
 drivers/media/platform/vsp1/vsp1_entity.c | 12 ++++++++++++
 drivers/media/platform/vsp1/vsp1_entity.h |  2 ++
 6 files changed, 20 insertions(+), 32 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1.h b/drivers/media/platform/vsp1/vsp1.h
index 910d6b8e8b50..bea232820ead 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -85,8 +85,6 @@ struct vsp1_device {
 	struct media_entity_operations media_ops;
 
 	struct vsp1_drm *drm;
-
-	bool use_dl;
 };
 
 int vsp1_device_get(struct vsp1_device *vsp1);
@@ -104,14 +102,4 @@ static inline void vsp1_write(struct vsp1_device *vsp1, u32 reg, u32 data)
 	iowrite32(data, vsp1->mmio + reg);
 }
 
-#include "vsp1_dl.h"
-
-static inline void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
-{
-	if (e->vsp1->use_dl)
-		vsp1_dl_add(e, reg, data);
-	else
-		vsp1_write(e->vsp1, reg, data);
-}
-
 #endif /* __VSP1_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 7dc27ac6bd02..3bdd002a9c80 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -18,7 +18,6 @@
 
 #include "vsp1.h"
 #include "vsp1_dl.h"
-#include "vsp1_pipe.h"
 
 /*
  * Global resources
@@ -129,10 +128,8 @@ void vsp1_dl_begin(struct vsp1_dl *dl)
 	list->reg_count = 0;
 }
 
-void vsp1_dl_add(struct vsp1_entity *e, u32 reg, u32 data)
+void vsp1_dl_add(struct vsp1_dl *dl, u32 reg, u32 data)
 {
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&e->subdev.entity);
-	struct vsp1_dl *dl = pipe->dl;
 	struct vsp1_dl_list *list = dl->lists.write;
 
 	list->body[list->reg_count].addr = reg;
diff --git a/drivers/media/platform/vsp1/vsp1_dl.h b/drivers/media/platform/vsp1/vsp1_dl.h
index 448c4250e54c..f4116ca59c28 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.h
+++ b/drivers/media/platform/vsp1/vsp1_dl.h
@@ -13,7 +13,7 @@
 #ifndef __VSP1_DL_H__
 #define __VSP1_DL_H__
 
-#include "vsp1_entity.h"
+#include <linux/types.h>
 
 struct vsp1_device;
 struct vsp1_dl;
@@ -25,18 +25,10 @@ void vsp1_dl_setup(struct vsp1_device *vsp1);
 
 void vsp1_dl_reset(struct vsp1_dl *dl);
 void vsp1_dl_begin(struct vsp1_dl *dl);
-void vsp1_dl_add(struct vsp1_entity *e, u32 reg, u32 data);
+void vsp1_dl_add(struct vsp1_dl *dl, u32 reg, u32 data);
 void vsp1_dl_commit(struct vsp1_dl *dl);
 
 void vsp1_dl_irq_display_start(struct vsp1_dl *dl);
 void vsp1_dl_irq_frame_end(struct vsp1_dl *dl);
 
-static inline void vsp1_dl_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
-{
-	if (e->vsp1->use_dl)
-		vsp1_dl_add(e, reg, data);
-	else
-		vsp1_write(e->vsp1, reg, data);
-}
-
 #endif /* __VSP1_DL_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index da43e3f35610..58632d766a2a 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -387,13 +387,10 @@ static int vsp1_create_entities(struct vsp1_device *vsp1)
 	/* Register subdev nodes if the userspace API is enabled or initialize
 	 * the DRM pipeline otherwise.
 	 */
-	if (vsp1->info->uapi) {
-		vsp1->use_dl = false;
+	if (vsp1->info->uapi)
 		ret = v4l2_device_register_subdev_nodes(&vsp1->v4l2_dev);
-	} else {
-		vsp1->use_dl = true;
+	else
 		ret = vsp1_drm_init(vsp1);
-	}
 	if (ret < 0)
 		goto done;
 
@@ -465,7 +462,7 @@ static int vsp1_device_init(struct vsp1_device *vsp1)
 	vsp1_write(vsp1, VI6_DPR_HGT_SMPPT, (7 << VI6_DPR_SMPPT_TGW_SHIFT) |
 		   (VI6_DPR_NODE_UNUSED << VI6_DPR_SMPPT_PT_SHIFT));
 
-	if (vsp1->use_dl)
+	if (!vsp1->info->uapi)
 		vsp1_dl_setup(vsp1);
 
 	return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 20a78fbd3691..4006f0d28bac 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -19,7 +19,19 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_entity.h"
+#include "vsp1_pipe.h"
+
+void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
+{
+	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&e->subdev.entity);
+
+	if (pipe->dl)
+		vsp1_dl_add(pipe->dl, reg, data);
+	else
+		vsp1_write(e->vsp1, reg, data);
+}
 
 bool vsp1_entity_is_streaming(struct vsp1_entity *entity)
 {
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 83570dfde8ec..311d5b64c9a5 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -103,4 +103,6 @@ int vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming);
 
 void vsp1_entity_route_setup(struct vsp1_entity *source);
 
+void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data);
+
 #endif /* __VSP1_ENTITY_H__ */
-- 
2.7.3

[PATCH v2 10/54] v4l: vsp1: Always setup the display list

[Laurent Pinchart]

Make sure display list usage is correctly disabled by always setting up
the corresponding registers, including when the display list feature
isn't used.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_dl.c  | 7 +++----
 drivers/media/platform/vsp1/vsp1_drv.c | 3 +--
 2 files changed, 4 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 3bdd002a9c80..7f9fe09af92d 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -243,15 +243,14 @@ done:
 
 void vsp1_dl_setup(struct vsp1_device *vsp1)
 {
-	u32 ctrl = (256 << VI6_DL_CTRL_AR_WAIT_SHIFT)
-		 | VI6_DL_CTRL_DC2 | VI6_DL_CTRL_DC1 | VI6_DL_CTRL_DC0
-		 | VI6_DL_CTRL_DLE;
+	u32 ctrl = (256 << VI6_DL_CTRL_AR_WAIT_SHIFT);
 
 	/* The DRM pipeline operates with header-less display lists in
 	 * Continuous Frame Mode.
 	 */
 	if (vsp1->drm)
-		ctrl |= VI6_DL_CTRL_CFM0 | VI6_DL_CTRL_NH0;
+		ctrl |= VI6_DL_CTRL_DC2 | VI6_DL_CTRL_DC1 | VI6_DL_CTRL_DC0
+		     |  VI6_DL_CTRL_DLE | VI6_DL_CTRL_CFM0 | VI6_DL_CTRL_NH0;
 
 	vsp1_write(vsp1, VI6_DL_CTRL, ctrl);
 	vsp1_write(vsp1, VI6_DL_SWAP, VI6_DL_SWAP_LWS);
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index 58632d766a2a..d657949bac3b 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -462,8 +462,7 @@ static int vsp1_device_init(struct vsp1_device *vsp1)
 	vsp1_write(vsp1, VI6_DPR_HGT_SMPPT, (7 << VI6_DPR_SMPPT_TGW_SHIFT) |
 		   (VI6_DPR_NODE_UNUSED << VI6_DPR_SMPPT_PT_SHIFT));
 
-	if (!vsp1->info->uapi)
-		vsp1_dl_setup(vsp1);
+	vsp1_dl_setup(vsp1);
 
 	return 0;
 }
-- 
2.7.3

[PATCH v2 11/54] v4l: vsp1: Simplify frame end processing

[Laurent Pinchart]

The DRM pipeline, as it runs in automatic restart mode, never sees the
pipeline state set to VSP1_PIPELINE_STOPPING or VSP1_PIPELINE_STOPPED
when running the frame end interrupt handler. We can thus skip the
checks various checks in the handler and return immediately.

Similarly the DRM frame end handler calls vsp1_pipeline_run()
unnecessarily, as the state there is never VSP1_PIPELINE_STOPPED. Remove
the function call and the frame end handler is it's now empty.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_drm.c  | 15 ---------------
 drivers/media/platform/vsp1/vsp1_pipe.c |  9 ++++++---
 2 files changed, 6 insertions(+), 18 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 8cf7c19f4344..9ecba4c1332e 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -27,20 +27,6 @@
 #include "vsp1_rwpf.h"
 
 /* -----------------------------------------------------------------------------
- * Runtime Handling
- */
-
-static void vsp1_drm_pipeline_frame_end(struct vsp1_pipeline *pipe)
-{
-	unsigned long flags;
-
-	spin_lock_irqsave(&pipe->irqlock, flags);
-	if (pipe->num_inputs)
-		vsp1_pipeline_run(pipe);
-	spin_unlock_irqrestore(&pipe->irqlock, flags);
-}
-
-/* -----------------------------------------------------------------------------
  * DU Driver API
  */
 
@@ -569,7 +555,6 @@ int vsp1_drm_init(struct vsp1_device *vsp1)
 	pipe = &vsp1->drm->pipe;
 
 	vsp1_pipeline_init(pipe);
-	pipe->frame_end = vsp1_drm_pipeline_frame_end;
 
 	/* The DRM pipeline is static, add entities manually. */
 	for (i = 0; i < vsp1->info->rpf_count; ++i) {
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 6659f06b1643..78096122a22d 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -289,7 +289,8 @@ void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
 		vsp1_dl_irq_frame_end(pipe->dl);
 
 	/* Signal frame end to the pipeline handler. */
-	pipe->frame_end(pipe);
+	if (pipe->frame_end)
+		pipe->frame_end(pipe);
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
@@ -298,8 +299,10 @@ void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
 	/* When using display lists in continuous frame mode the pipeline is
 	 * automatically restarted by the hardware.
 	 */
-	if (!pipe->dl)
-		pipe->state = VSP1_PIPELINE_STOPPED;
+	if (pipe->dl)
+		goto done;
+
+	pipe->state = VSP1_PIPELINE_STOPPED;
 
 	/* If a stop has been requested, mark the pipeline as stopped and
 	 * return.
-- 
2.7.3

[PATCH v2 12/54] v4l: vsp1: Split display list manager from display list

[Laurent Pinchart]

This clarifies the API and prepares display list support for being used
to implement the request API.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1.h        |   1 -
 drivers/media/platform/vsp1/vsp1_dl.c     | 264 ++++++++++++++----------------
 drivers/media/platform/vsp1/vsp1_dl.h     |  40 +++--
 drivers/media/platform/vsp1/vsp1_drm.c    |  32 ++--
 drivers/media/platform/vsp1/vsp1_drm.h    |  12 +-
 drivers/media/platform/vsp1/vsp1_drv.c    |  11 +-
 drivers/media/platform/vsp1/vsp1_entity.c |   2 +-
 drivers/media/platform/vsp1/vsp1_pipe.c   |  13 +-
 drivers/media/platform/vsp1/vsp1_pipe.h   |   5 +-
 9 files changed, 194 insertions(+), 186 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1.h b/drivers/media/platform/vsp1/vsp1.h
index bea232820ead..dae987a11a70 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -26,7 +26,6 @@
 struct clk;
 struct device;
 
-struct vsp1_dl;
 struct vsp1_drm;
 struct vsp1_entity;
 struct vsp1_platform_data;
diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 7f9fe09af92d..72fb667814eb 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -37,117 +37,109 @@ struct vsp1_dl_entry {
 } __attribute__((__packed__));
 
 struct vsp1_dl_list {
-	size_t size;
-	int reg_count;
+	struct list_head list;
 
-	bool in_use;
+	struct vsp1_dl_manager *dlm;
 
 	struct vsp1_dl_entry *body;
 	dma_addr_t dma;
-};
-
-/**
- * struct vsp1_dl - Display List manager
- * @vsp1: the VSP1 device
- * @lock: protects the active, queued and pending lists
- * @lists.all: array of all allocate display lists
- * @lists.active: list currently being processed (loaded) by hardware
- * @lists.queued: list queued to the hardware (written to the DL registers)
- * @lists.pending: list waiting to be queued to the hardware
- * @lists.write: list being written to by software
- */
-struct vsp1_dl {
-	struct vsp1_device *vsp1;
-
-	spinlock_t lock;
-
 	size_t size;
-	dma_addr_t dma;
-	void *mem;
 
-	struct {
-		struct vsp1_dl_list all[VSP1_DL_NUM_LISTS];
-
-		struct vsp1_dl_list *active;
-		struct vsp1_dl_list *queued;
-		struct vsp1_dl_list *pending;
-		struct vsp1_dl_list *write;
-	} lists;
+	int reg_count;
 };
 
 /* -----------------------------------------------------------------------------
  * Display List Transaction Management
  */
 
-static void vsp1_dl_free_list(struct vsp1_dl_list *list)
+static struct vsp1_dl_list *vsp1_dl_list_alloc(struct vsp1_dl_manager *dlm)
 {
-	if (!list)
-		return;
+	struct vsp1_dl_list *dl;
 
-	list->in_use = false;
-}
+	dl = kzalloc(sizeof(*dl), GFP_KERNEL);
+	if (!dl)
+		return NULL;
 
-void vsp1_dl_reset(struct vsp1_dl *dl)
-{
-	unsigned int i;
+	dl->dlm = dlm;
+	dl->size = VSP1_DL_BODY_SIZE;
+
+	dl->body = dma_alloc_writecombine(dlm->vsp1->dev, dl->size, &dl->dma,
+					  GFP_KERNEL);
+	if (!dl->body) {
+		kfree(dl);
+		return NULL;
+	}
 
-	dl->lists.active = NULL;
-	dl->lists.queued = NULL;
-	dl->lists.pending = NULL;
-	dl->lists.write = NULL;
+	return dl;
+}
 
-	for (i = 0; i < ARRAY_SIZE(dl->lists.all); ++i)
-		dl->lists.all[i].in_use = false;
+static void vsp1_dl_list_free(struct vsp1_dl_list *dl)
+{
+	dma_free_writecombine(dl->dlm->vsp1->dev, dl->size, dl->body, dl->dma);
+	kfree(dl);
 }
 
-void vsp1_dl_begin(struct vsp1_dl *dl)
+/**
+ * vsp1_dl_list_get - Get a free display list
+ * @dlm: The display list manager
+ *
+ * Get a display list from the pool of free lists and return it.
+ *
+ * This function must be called without the display list manager lock held.
+ */
+struct vsp1_dl_list *vsp1_dl_list_get(struct vsp1_dl_manager *dlm)
 {
-	struct vsp1_dl_list *list = NULL;
+	struct vsp1_dl_list *dl = NULL;
 	unsigned long flags;
-	unsigned int i;
 
-	spin_lock_irqsave(&dl->lock, flags);
+	spin_lock_irqsave(&dlm->lock, flags);
 
-	for (i = 0; i < ARRAY_SIZE(dl->lists.all); ++i) {
-		if (!dl->lists.all[i].in_use) {
-			list = &dl->lists.all[i];
-			break;
-		}
+	if (!list_empty(&dlm->free)) {
+		dl = list_first_entry(&dlm->free, struct vsp1_dl_list, list);
+		list_del(&dl->list);
 	}
 
-	if (!list) {
-		list = dl->lists.pending;
-		dl->lists.pending = NULL;
-	}
+	spin_unlock_irqrestore(&dlm->lock, flags);
+
+	return dl;
+}
 
-	spin_unlock_irqrestore(&dl->lock, flags);
+/**
+ * vsp1_dl_list_put - Release a display list
+ * @dl: The display list
+ *
+ * Release the display list and return it to the pool of free lists.
+ *
+ * This function must be called with the display list manager lock held.
+ *
+ * Passing a NULL pointer to this function is safe, in that case no operation
+ * will be performed.
+ */
+void vsp1_dl_list_put(struct vsp1_dl_list *dl)
+{
+	if (!dl)
+		return;
 
-	dl->lists.write = list;
+	dl->reg_count = 0;
 
-	list->in_use = true;
-	list->reg_count = 0;
+	list_add_tail(&dl->list, &dl->dlm->free);
 }
 
-void vsp1_dl_add(struct vsp1_dl *dl, u32 reg, u32 data)
+void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg, u32 data)
 {
-	struct vsp1_dl_list *list = dl->lists.write;
-
-	list->body[list->reg_count].addr = reg;
-	list->body[list->reg_count].data = data;
-	list->reg_count++;
+	dl->body[dl->reg_count].addr = reg;
+	dl->body[dl->reg_count].data = data;
+	dl->reg_count++;
 }
 
-void vsp1_dl_commit(struct vsp1_dl *dl)
+void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
 {
-	struct vsp1_device *vsp1 = dl->vsp1;
-	struct vsp1_dl_list *list;
+	struct vsp1_dl_manager *dlm = dl->dlm;
+	struct vsp1_device *vsp1 = dlm->vsp1;
 	unsigned long flags;
 	bool update;
 
-	list = dl->lists.write;
-	dl->lists.write = NULL;
-
-	spin_lock_irqsave(&dl->lock, flags);
+	spin_lock_irqsave(&dlm->lock, flags);
 
 	/* Once the UPD bit has been set the hardware can start processing the
 	 * display list at any time and we can't touch the address and size
@@ -156,8 +148,8 @@ void vsp1_dl_commit(struct vsp1_dl *dl)
 	 */
 	update = !!(vsp1_read(vsp1, VI6_DL_BODY_SIZE) & VI6_DL_BODY_SIZE_UPD);
 	if (update) {
-		vsp1_dl_free_list(dl->lists.pending);
-		dl->lists.pending = list;
+		vsp1_dl_list_put(dlm->pending);
+		dlm->pending = dl;
 		goto done;
 	}
 
@@ -165,42 +157,44 @@ void vsp1_dl_commit(struct vsp1_dl *dl)
 	 * The UPD bit will be cleared by the device when the display list is
 	 * processed.
 	 */
-	vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), list->dma);
+	vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->dma);
 	vsp1_write(vsp1, VI6_DL_BODY_SIZE, VI6_DL_BODY_SIZE_UPD |
-		   (list->reg_count * 8));
+		   (dl->reg_count * 8));
 
-	vsp1_dl_free_list(dl->lists.queued);
-	dl->lists.queued = list;
+	vsp1_dl_list_put(dlm->queued);
+	dlm->queued = dl;
 
 done:
-	spin_unlock_irqrestore(&dl->lock, flags);
+	spin_unlock_irqrestore(&dlm->lock, flags);
 }
 
 /* -----------------------------------------------------------------------------
- * Interrupt Handling
+ * Display List Manager
  */
 
-void vsp1_dl_irq_display_start(struct vsp1_dl *dl)
+/* Interrupt Handling */
+void vsp1_dlm_irq_display_start(struct vsp1_dl_manager *dlm)
 {
-	spin_lock(&dl->lock);
+	spin_lock(&dlm->lock);
 
 	/* The display start interrupt signals the end of the display list
 	 * processing by the device. The active display list, if any, won't be
 	 * accessed anymore and can be reused.
 	 */
-	if (dl->lists.active) {
-		vsp1_dl_free_list(dl->lists.active);
-		dl->lists.active = NULL;
-	}
+	vsp1_dl_list_put(dlm->active);
+	dlm->active = NULL;
 
-	spin_unlock(&dl->lock);
+	spin_unlock(&dlm->lock);
 }
 
-void vsp1_dl_irq_frame_end(struct vsp1_dl *dl)
+void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
 {
-	struct vsp1_device *vsp1 = dl->vsp1;
+	struct vsp1_device *vsp1 = dlm->vsp1;
 
-	spin_lock(&dl->lock);
+	spin_lock(&dlm->lock);
+
+	vsp1_dl_list_put(dlm->active);
+	dlm->active = NULL;
 
 	/* The UPD bit set indicates that the commit operation raced with the
 	 * interrupt and occurred after the frame end event and UPD clear but
@@ -213,35 +207,31 @@ void vsp1_dl_irq_frame_end(struct vsp1_dl *dl)
 	/* The device starts processing the queued display list right after the
 	 * frame end interrupt. The display list thus becomes active.
 	 */
-	if (dl->lists.queued) {
-		WARN_ON(dl->lists.active);
-		dl->lists.active = dl->lists.queued;
-		dl->lists.queued = NULL;
+	if (dlm->queued) {
+		dlm->active = dlm->queued;
+		dlm->queued = NULL;
 	}
 
 	/* Now that the UPD bit has been cleared we can queue the next display
 	 * list to the hardware if one has been prepared.
 	 */
-	if (dl->lists.pending) {
-		struct vsp1_dl_list *list = dl->lists.pending;
+	if (dlm->pending) {
+		struct vsp1_dl_list *dl = dlm->pending;
 
-		vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), list->dma);
+		vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->dma);
 		vsp1_write(vsp1, VI6_DL_BODY_SIZE, VI6_DL_BODY_SIZE_UPD |
-			   (list->reg_count * 8));
+			   (dl->reg_count * 8));
 
-		dl->lists.queued = list;
-		dl->lists.pending = NULL;
+		dlm->queued = dl;
+		dlm->pending = NULL;
 	}
 
 done:
-	spin_unlock(&dl->lock);
+	spin_unlock(&dlm->lock);
 }
 
-/* -----------------------------------------------------------------------------
- * Hardware Setup
- */
-
-void vsp1_dl_setup(struct vsp1_device *vsp1)
+/* Hardware Setup */
+void vsp1_dlm_setup(struct vsp1_device *vsp1)
 {
 	u32 ctrl = (256 << VI6_DL_CTRL_AR_WAIT_SHIFT);
 
@@ -256,46 +246,46 @@ void vsp1_dl_setup(struct vsp1_device *vsp1)
 	vsp1_write(vsp1, VI6_DL_SWAP, VI6_DL_SWAP_LWS);
 }
 
-/* -----------------------------------------------------------------------------
- * Initialization and Cleanup
- */
+void vsp1_dlm_reset(struct vsp1_dl_manager *dlm)
+{
+	vsp1_dl_list_put(dlm->active);
+	vsp1_dl_list_put(dlm->queued);
+	vsp1_dl_list_put(dlm->pending);
+
+	dlm->active = NULL;
+	dlm->queued = NULL;
+	dlm->pending = NULL;
+}
 
-struct vsp1_dl *vsp1_dl_create(struct vsp1_device *vsp1)
+int vsp1_dlm_init(struct vsp1_device *vsp1, struct vsp1_dl_manager *dlm,
+		  unsigned int prealloc)
 {
-	struct vsp1_dl *dl;
 	unsigned int i;
 
-	dl = kzalloc(sizeof(*dl), GFP_KERNEL);
-	if (!dl)
-		return NULL;
-
-	spin_lock_init(&dl->lock);
+	dlm->vsp1 = vsp1;
 
-	dl->vsp1 = vsp1;
-	dl->size = VSP1_DL_BODY_SIZE * ARRAY_SIZE(dl->lists.all);
+	spin_lock_init(&dlm->lock);
+	INIT_LIST_HEAD(&dlm->free);
 
-	dl->mem = dma_alloc_writecombine(vsp1->dev, dl->size, &dl->dma,
-					 GFP_KERNEL);
-	if (!dl->mem) {
-		kfree(dl);
-		return NULL;
-	}
+	for (i = 0; i < prealloc; ++i) {
+		struct vsp1_dl_list *dl;
 
-	for (i = 0; i < ARRAY_SIZE(dl->lists.all); ++i) {
-		struct vsp1_dl_list *list = &dl->lists.all[i];
+		dl = vsp1_dl_list_alloc(dlm);
+		if (!dl)
+			return -ENOMEM;
 
-		list->size = VSP1_DL_BODY_SIZE;
-		list->reg_count = 0;
-		list->in_use = false;
-		list->dma = dl->dma + VSP1_DL_BODY_SIZE * i;
-		list->body = dl->mem + VSP1_DL_BODY_SIZE * i;
+		list_add_tail(&dl->list, &dlm->free);
 	}
 
-	return dl;
+	return 0;
 }
 
-void vsp1_dl_destroy(struct vsp1_dl *dl)
+void vsp1_dlm_cleanup(struct vsp1_dl_manager *dlm)
 {
-	dma_free_writecombine(dl->vsp1->dev, dl->size, dl->mem, dl->dma);
-	kfree(dl);
+	struct vsp1_dl_list *dl, *next;
+
+	list_for_each_entry_safe(dl, next, &dlm->free, list) {
+		list_del(&dl->list);
+		vsp1_dl_list_free(dl);
+	}
 }
diff --git a/drivers/media/platform/vsp1/vsp1_dl.h b/drivers/media/platform/vsp1/vsp1_dl.h
index f4116ca59c28..caa6a85f6825 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.h
+++ b/drivers/media/platform/vsp1/vsp1_dl.h
@@ -16,19 +16,39 @@
 #include <linux/types.h>
 
 struct vsp1_device;
-struct vsp1_dl;
+struct vsp1_dl_list;
 
-struct vsp1_dl *vsp1_dl_create(struct vsp1_device *vsp1);
-void vsp1_dl_destroy(struct vsp1_dl *dl);
+/**
+ * struct vsp1_dl_manager - Display List manager
+ * @vsp1: the VSP1 device
+ * @lock: protects the active, queued and pending lists
+ * @free: array of all free display lists
+ * @active: list currently being processed (loaded) by hardware
+ * @queued: list queued to the hardware (written to the DL registers)
+ * @pending: list waiting to be queued to the hardware
+ */
+struct vsp1_dl_manager {
+	struct vsp1_device *vsp1;
+
+	spinlock_t lock;
+	struct list_head free;
+	struct vsp1_dl_list *active;
+	struct vsp1_dl_list *queued;
+	struct vsp1_dl_list *pending;
+};
 
-void vsp1_dl_setup(struct vsp1_device *vsp1);
+void vsp1_dlm_setup(struct vsp1_device *vsp1);
 
-void vsp1_dl_reset(struct vsp1_dl *dl);
-void vsp1_dl_begin(struct vsp1_dl *dl);
-void vsp1_dl_add(struct vsp1_dl *dl, u32 reg, u32 data);
-void vsp1_dl_commit(struct vsp1_dl *dl);
+int vsp1_dlm_init(struct vsp1_device *vsp1, struct vsp1_dl_manager *dlm,
+		  unsigned int prealloc);
+void vsp1_dlm_cleanup(struct vsp1_dl_manager *dlm);
+void vsp1_dlm_reset(struct vsp1_dl_manager *dlm);
+void vsp1_dlm_irq_display_start(struct vsp1_dl_manager *dlm);
+void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm);
 
-void vsp1_dl_irq_display_start(struct vsp1_dl *dl);
-void vsp1_dl_irq_frame_end(struct vsp1_dl *dl);
+struct vsp1_dl_list *vsp1_dl_list_get(struct vsp1_dl_manager *dlm);
+void vsp1_dl_list_put(struct vsp1_dl_list *dl);
+void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg, u32 data);
+void vsp1_dl_list_commit(struct vsp1_dl_list *dl);
 
 #endif /* __VSP1_DL_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 9ecba4c1332e..a8cd74335f20 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -26,6 +26,18 @@
 #include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
 
+
+/* -----------------------------------------------------------------------------
+ * Interrupt Handling
+ */
+
+void vsp1_drm_frame_end(struct vsp1_pipeline *pipe)
+{
+	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+
+	vsp1_dlm_irq_frame_end(&vsp1->drm->dlm);
+}
+
 /* -----------------------------------------------------------------------------
  * DU Driver API
  */
@@ -89,6 +101,7 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width,
 
 		pipe->num_inputs = 0;
 
+		vsp1_dlm_reset(&vsp1->drm->dlm);
 		vsp1_device_put(vsp1);
 
 		dev_dbg(vsp1->dev, "%s: pipeline disabled\n", __func__);
@@ -96,8 +109,6 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width,
 		return 0;
 	}
 
-	vsp1_dl_reset(vsp1->drm->dl);
-
 	/* Configure the format at the BRU sinks and propagate it through the
 	 * pipeline.
 	 */
@@ -217,7 +228,7 @@ void vsp1_du_atomic_begin(struct device *dev)
 	spin_unlock_irqrestore(&pipe->irqlock, flags);
 
 	/* Prepare the display list. */
-	vsp1_dl_begin(vsp1->drm->dl);
+	pipe->dl = vsp1_dl_list_get(&vsp1->drm->dlm);
 }
 EXPORT_SYMBOL_GPL(vsp1_du_atomic_begin);
 
@@ -467,7 +478,8 @@ void vsp1_du_atomic_flush(struct device *dev)
 		}
 	}
 
-	vsp1_dl_commit(vsp1->drm->dl);
+	vsp1_dl_list_commit(pipe->dl);
+	pipe->dl = NULL;
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
@@ -543,18 +555,20 @@ int vsp1_drm_init(struct vsp1_device *vsp1)
 {
 	struct vsp1_pipeline *pipe;
 	unsigned int i;
+	int ret;
 
 	vsp1->drm = devm_kzalloc(vsp1->dev, sizeof(*vsp1->drm), GFP_KERNEL);
 	if (!vsp1->drm)
 		return -ENOMEM;
 
-	vsp1->drm->dl = vsp1_dl_create(vsp1);
-	if (!vsp1->drm->dl)
-		return -ENOMEM;
+	ret = vsp1_dlm_init(vsp1, &vsp1->drm->dlm, 4);
+	if (ret < 0)
+		return ret;
 
 	pipe = &vsp1->drm->pipe;
 
 	vsp1_pipeline_init(pipe);
+	pipe->frame_end = vsp1_drm_frame_end;
 
 	/* The DRM pipeline is static, add entities manually. */
 	for (i = 0; i < vsp1->info->rpf_count; ++i) {
@@ -571,12 +585,10 @@ int vsp1_drm_init(struct vsp1_device *vsp1)
 	pipe->lif = &vsp1->lif->entity;
 	pipe->output = vsp1->wpf[0];
 
-	pipe->dl = vsp1->drm->dl;
-
 	return 0;
 }
 
 void vsp1_drm_cleanup(struct vsp1_device *vsp1)
 {
-	vsp1_dl_destroy(vsp1->drm->dl);
+	vsp1_dlm_cleanup(&vsp1->drm->dlm);
 }
diff --git a/drivers/media/platform/vsp1/vsp1_drm.h b/drivers/media/platform/vsp1/vsp1_drm.h
index 7704038c3add..5ef32cff9601 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.h
+++ b/drivers/media/platform/vsp1/vsp1_drm.h
@@ -13,26 +13,30 @@
 #ifndef __VSP1_DRM_H__
 #define __VSP1_DRM_H__
 
+#include "vsp1_dl.h"
 #include "vsp1_pipe.h"
 
-struct vsp1_dl;
-
 /**
  * vsp1_drm - State for the API exposed to the DRM driver
- * @dl: display list for DRM pipeline operation
  * @pipe: the VSP1 pipeline used for display
  * @num_inputs: number of active pipeline inputs at the beginning of an update
  * @update: the pipeline configuration has been updated
+ * @dlm: display list manager used for DRM operation
  */
 struct vsp1_drm {
-	struct vsp1_dl *dl;
 	struct vsp1_pipeline pipe;
 	unsigned int num_inputs;
 	bool update;
+	struct vsp1_dl_manager dlm;
 };
 
 int vsp1_drm_init(struct vsp1_device *vsp1);
 void vsp1_drm_cleanup(struct vsp1_device *vsp1);
 int vsp1_drm_create_links(struct vsp1_device *vsp1);
 
+static inline void vsp1_drm_display_start(struct vsp1_device *vsp1)
+{
+	vsp1_dlm_irq_display_start(&vsp1->drm->dlm);
+}
+
 #endif /* __VSP1_DRM_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index d657949bac3b..bfdc01c9172d 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -68,14 +68,7 @@ static irqreturn_t vsp1_irq_handler(int irq, void *data)
 	vsp1_write(vsp1, VI6_DISP_IRQ_STA, ~status & VI6_DISP_IRQ_STA_DST);
 
 	if (status & VI6_DISP_IRQ_STA_DST) {
-		struct vsp1_rwpf *wpf = vsp1->wpf[0];
-		struct vsp1_pipeline *pipe;
-
-		if (wpf) {
-			pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
-			vsp1_pipeline_display_start(pipe);
-		}
-
+		vsp1_drm_display_start(vsp1);
 		ret = IRQ_HANDLED;
 	}
 
@@ -462,7 +455,7 @@ static int vsp1_device_init(struct vsp1_device *vsp1)
 	vsp1_write(vsp1, VI6_DPR_HGT_SMPPT, (7 << VI6_DPR_SMPPT_TGW_SHIFT) |
 		   (VI6_DPR_NODE_UNUSED << VI6_DPR_SMPPT_PT_SHIFT));
 
-	vsp1_dl_setup(vsp1);
+	vsp1_dlm_setup(vsp1);
 
 	return 0;
 }
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 4006f0d28bac..83689588900a 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -28,7 +28,7 @@ void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&e->subdev.entity);
 
 	if (pipe->dl)
-		vsp1_dl_add(pipe->dl, reg, data);
+		vsp1_dl_list_write(pipe->dl, reg, data);
 	else
 		vsp1_write(e->vsp1, reg, data);
 }
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 78096122a22d..cb67b8f80635 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -226,7 +226,7 @@ int vsp1_pipeline_stop(struct vsp1_pipeline *pipe)
 	unsigned long flags;
 	int ret;
 
-	if (pipe->dl) {
+	if (pipe->lif) {
 		/* When using display lists in continuous frame mode the only
 		 * way to stop the pipeline is to reset the hardware.
 		 */
@@ -271,12 +271,6 @@ bool vsp1_pipeline_ready(struct vsp1_pipeline *pipe)
 	return pipe->buffers_ready == mask;
 }
 
-void vsp1_pipeline_display_start(struct vsp1_pipeline *pipe)
-{
-	if (pipe->dl)
-		vsp1_dl_irq_display_start(pipe->dl);
-}
-
 void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
 {
 	enum vsp1_pipeline_state state;
@@ -285,9 +279,6 @@ void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
 	if (pipe == NULL)
 		return;
 
-	if (pipe->dl)
-		vsp1_dl_irq_frame_end(pipe->dl);
-
 	/* Signal frame end to the pipeline handler. */
 	if (pipe->frame_end)
 		pipe->frame_end(pipe);
@@ -299,7 +290,7 @@ void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
 	/* When using display lists in continuous frame mode the pipeline is
 	 * automatically restarted by the hardware.
 	 */
-	if (pipe->dl)
+	if (pipe->lif)
 		goto done;
 
 	pipe->state = VSP1_PIPELINE_STOPPED;
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h b/drivers/media/platform/vsp1/vsp1_pipe.h
index b2f3a8a896c9..f4bdfc943add 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -19,7 +19,7 @@
 
 #include <media/media-entity.h>
 
-struct vsp1_dl;
+struct vsp1_dl_list;
 struct vsp1_rwpf;
 
 /*
@@ -100,7 +100,7 @@ struct vsp1_pipeline {
 
 	struct list_head entities;
 
-	struct vsp1_dl *dl;
+	struct vsp1_dl_list *dl;
 };
 
 static inline struct vsp1_pipeline *to_vsp1_pipeline(struct media_entity *e)
@@ -119,7 +119,6 @@ bool vsp1_pipeline_stopped(struct vsp1_pipeline *pipe);
 int vsp1_pipeline_stop(struct vsp1_pipeline *pipe);
 bool vsp1_pipeline_ready(struct vsp1_pipeline *pipe);
 
-void vsp1_pipeline_display_start(struct vsp1_pipeline *pipe);
 void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe);
 
 void vsp1_pipeline_propagate_alpha(struct vsp1_pipeline *pipe,
-- 
2.7.3

[PATCH v2 13/54] v4l: vsp1: Store the display list manager in the WPF

[Laurent Pinchart]

Each WPF can process display lists independently, move the manager to
the WPF to reflect that and prepare for display list support for non-DRM
pipelines.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_dl.c     | 37 ++++++++++++++++++++++++++-----
 drivers/media/platform/vsp1/vsp1_dl.h     | 26 ++++------------------
 drivers/media/platform/vsp1/vsp1_drm.c    | 19 +++++++---------
 drivers/media/platform/vsp1/vsp1_drm.h    |  8 +------
 drivers/media/platform/vsp1/vsp1_entity.c |  2 ++
 drivers/media/platform/vsp1/vsp1_entity.h |  2 ++
 drivers/media/platform/vsp1/vsp1_rwpf.h   |  3 +++
 drivers/media/platform/vsp1/vsp1_wpf.c    | 18 +++++++++++++++
 8 files changed, 70 insertions(+), 45 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 72fb667814eb..0b2896c04f4f 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -48,6 +48,25 @@ struct vsp1_dl_list {
 	int reg_count;
 };
 
+/**
+ * struct vsp1_dl_manager - Display List manager
+ * @vsp1: the VSP1 device
+ * @lock: protects the active, queued and pending lists
+ * @free: array of all free display lists
+ * @active: list currently being processed (loaded) by hardware
+ * @queued: list queued to the hardware (written to the DL registers)
+ * @pending: list waiting to be queued to the hardware
+ */
+struct vsp1_dl_manager {
+	struct vsp1_device *vsp1;
+
+	spinlock_t lock;
+	struct list_head free;
+	struct vsp1_dl_list *active;
+	struct vsp1_dl_list *queued;
+	struct vsp1_dl_list *pending;
+};
+
 /* -----------------------------------------------------------------------------
  * Display List Transaction Management
  */
@@ -257,11 +276,16 @@ void vsp1_dlm_reset(struct vsp1_dl_manager *dlm)
 	dlm->pending = NULL;
 }
 
-int vsp1_dlm_init(struct vsp1_device *vsp1, struct vsp1_dl_manager *dlm,
-		  unsigned int prealloc)
+struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
+					unsigned int prealloc)
 {
+	struct vsp1_dl_manager *dlm;
 	unsigned int i;
 
+	dlm = devm_kzalloc(vsp1->dev, sizeof(*dlm), GFP_KERNEL);
+	if (!dlm)
+		return NULL;
+
 	dlm->vsp1 = vsp1;
 
 	spin_lock_init(&dlm->lock);
@@ -272,18 +296,21 @@ int vsp1_dlm_init(struct vsp1_device *vsp1, struct vsp1_dl_manager *dlm,
 
 		dl = vsp1_dl_list_alloc(dlm);
 		if (!dl)
-			return -ENOMEM;
+			return NULL;
 
 		list_add_tail(&dl->list, &dlm->free);
 	}
 
-	return 0;
+	return dlm;
 }
 
-void vsp1_dlm_cleanup(struct vsp1_dl_manager *dlm)
+void vsp1_dlm_destroy(struct vsp1_dl_manager *dlm)
 {
 	struct vsp1_dl_list *dl, *next;
 
+	if (!dlm)
+		return;
+
 	list_for_each_entry_safe(dl, next, &dlm->free, list) {
 		list_del(&dl->list);
 		vsp1_dl_list_free(dl);
diff --git a/drivers/media/platform/vsp1/vsp1_dl.h b/drivers/media/platform/vsp1/vsp1_dl.h
index caa6a85f6825..46f7ae337374 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.h
+++ b/drivers/media/platform/vsp1/vsp1_dl.h
@@ -17,31 +17,13 @@
 
 struct vsp1_device;
 struct vsp1_dl_list;
-
-/**
- * struct vsp1_dl_manager - Display List manager
- * @vsp1: the VSP1 device
- * @lock: protects the active, queued and pending lists
- * @free: array of all free display lists
- * @active: list currently being processed (loaded) by hardware
- * @queued: list queued to the hardware (written to the DL registers)
- * @pending: list waiting to be queued to the hardware
- */
-struct vsp1_dl_manager {
-	struct vsp1_device *vsp1;
-
-	spinlock_t lock;
-	struct list_head free;
-	struct vsp1_dl_list *active;
-	struct vsp1_dl_list *queued;
-	struct vsp1_dl_list *pending;
-};
+struct vsp1_dl_manager;
 
 void vsp1_dlm_setup(struct vsp1_device *vsp1);
 
-int vsp1_dlm_init(struct vsp1_device *vsp1, struct vsp1_dl_manager *dlm,
-		  unsigned int prealloc);
-void vsp1_dlm_cleanup(struct vsp1_dl_manager *dlm);
+struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
+					unsigned int prealloc);
+void vsp1_dlm_destroy(struct vsp1_dl_manager *dlm);
 void vsp1_dlm_reset(struct vsp1_dl_manager *dlm);
 void vsp1_dlm_irq_display_start(struct vsp1_dl_manager *dlm);
 void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm);
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index a8cd74335f20..22f67360b750 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -31,11 +31,14 @@
  * Interrupt Handling
  */
 
-void vsp1_drm_frame_end(struct vsp1_pipeline *pipe)
+void vsp1_drm_display_start(struct vsp1_device *vsp1)
 {
-	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+	vsp1_dlm_irq_display_start(vsp1->drm->pipe.output->dlm);
+}
 
-	vsp1_dlm_irq_frame_end(&vsp1->drm->dlm);
+void vsp1_drm_frame_end(struct vsp1_pipeline *pipe)
+{
+	vsp1_dlm_irq_frame_end(pipe->output->dlm);
 }
 
 /* -----------------------------------------------------------------------------
@@ -101,7 +104,7 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width,
 
 		pipe->num_inputs = 0;
 
-		vsp1_dlm_reset(&vsp1->drm->dlm);
+		vsp1_dlm_reset(pipe->output->dlm);
 		vsp1_device_put(vsp1);
 
 		dev_dbg(vsp1->dev, "%s: pipeline disabled\n", __func__);
@@ -228,7 +231,7 @@ void vsp1_du_atomic_begin(struct device *dev)
 	spin_unlock_irqrestore(&pipe->irqlock, flags);
 
 	/* Prepare the display list. */
-	pipe->dl = vsp1_dl_list_get(&vsp1->drm->dlm);
+	pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
 }
 EXPORT_SYMBOL_GPL(vsp1_du_atomic_begin);
 
@@ -555,16 +558,11 @@ int vsp1_drm_init(struct vsp1_device *vsp1)
 {
 	struct vsp1_pipeline *pipe;
 	unsigned int i;
-	int ret;
 
 	vsp1->drm = devm_kzalloc(vsp1->dev, sizeof(*vsp1->drm), GFP_KERNEL);
 	if (!vsp1->drm)
 		return -ENOMEM;
 
-	ret = vsp1_dlm_init(vsp1, &vsp1->drm->dlm, 4);
-	if (ret < 0)
-		return ret;
-
 	pipe = &vsp1->drm->pipe;
 
 	vsp1_pipeline_init(pipe);
@@ -590,5 +588,4 @@ int vsp1_drm_init(struct vsp1_device *vsp1)
 
 void vsp1_drm_cleanup(struct vsp1_device *vsp1)
 {
-	vsp1_dlm_cleanup(&vsp1->drm->dlm);
 }
diff --git a/drivers/media/platform/vsp1/vsp1_drm.h b/drivers/media/platform/vsp1/vsp1_drm.h
index 5ef32cff9601..e9242f2c870e 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.h
+++ b/drivers/media/platform/vsp1/vsp1_drm.h
@@ -13,7 +13,6 @@
 #ifndef __VSP1_DRM_H__
 #define __VSP1_DRM_H__
 
-#include "vsp1_dl.h"
 #include "vsp1_pipe.h"
 
 /**
@@ -21,22 +20,17 @@
  * @pipe: the VSP1 pipeline used for display
  * @num_inputs: number of active pipeline inputs at the beginning of an update
  * @update: the pipeline configuration has been updated
- * @dlm: display list manager used for DRM operation
  */
 struct vsp1_drm {
 	struct vsp1_pipeline pipe;
 	unsigned int num_inputs;
 	bool update;
-	struct vsp1_dl_manager dlm;
 };
 
 int vsp1_drm_init(struct vsp1_device *vsp1);
 void vsp1_drm_cleanup(struct vsp1_device *vsp1);
 int vsp1_drm_create_links(struct vsp1_device *vsp1);
 
-static inline void vsp1_drm_display_start(struct vsp1_device *vsp1)
-{
-	vsp1_dlm_irq_display_start(&vsp1->drm->dlm);
-}
+void vsp1_drm_display_start(struct vsp1_device *vsp1);
 
 #endif /* __VSP1_DRM_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 83689588900a..a94f544dcc77 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -244,6 +244,8 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 
 void vsp1_entity_destroy(struct vsp1_entity *entity)
 {
+	if (entity->destroy)
+		entity->destroy(entity);
 	if (entity->subdev.ctrl_handler)
 		v4l2_ctrl_handler_free(entity->subdev.ctrl_handler);
 	media_entity_cleanup(&entity->subdev.entity);
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 311d5b64c9a5..259880e524fe 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -56,6 +56,8 @@ struct vsp1_route {
 struct vsp1_entity {
 	struct vsp1_device *vsp1;
 
+	void (*destroy)(struct vsp1_entity *);
+
 	enum vsp1_entity_type type;
 	unsigned int index;
 	const struct vsp1_route *route;
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 8e8235682ada..d04df39b2737 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -24,6 +24,7 @@
 #define RWPF_PAD_SOURCE				1
 
 struct v4l2_ctrl;
+struct vsp1_dl_manager;
 struct vsp1_rwpf;
 struct vsp1_video;
 
@@ -60,6 +61,8 @@ struct vsp1_rwpf {
 
 	unsigned int offsets[2];
 	dma_addr_t buf_addr[3];
+
+	struct vsp1_dl_manager *dlm;
 };
 
 static inline struct vsp1_rwpf *to_rwpf(struct v4l2_subdev *subdev)
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index d2735f09d1da..3640989b3fd5 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -16,6 +16,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
@@ -218,6 +219,13 @@ static const struct vsp1_rwpf_operations wpf_vdev_ops = {
  * Initialization and Cleanup
  */
 
+static void vsp1_wpf_destroy(struct vsp1_entity *entity)
+{
+	struct vsp1_rwpf *wpf = container_of(entity, struct vsp1_rwpf, entity);
+
+	vsp1_dlm_destroy(wpf->dlm);
+}
+
 struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 {
 	struct v4l2_subdev *subdev;
@@ -233,6 +241,7 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	wpf->max_width = WPF_MAX_WIDTH;
 	wpf->max_height = WPF_MAX_HEIGHT;
 
+	wpf->entity.destroy = vsp1_wpf_destroy;
 	wpf->entity.type = VSP1_ENTITY_WPF;
 	wpf->entity.index = index;
 
@@ -240,6 +249,15 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	if (ret < 0)
 		return ERR_PTR(ret);
 
+	/* Initialize the display list manager if the WPF is used for display */
+	if ((vsp1->info->features & VSP1_HAS_LIF) && index == 0) {
+		wpf->dlm = vsp1_dlm_create(vsp1, 4);
+		if (!wpf->dlm) {
+			ret = -ENOMEM;
+			goto error;
+		}
+	}
+
 	/* Initialize the V4L2 subdev. */
 	subdev = &wpf->entity.subdev;
 	v4l2_subdev_init(subdev, &wpf_ops);
-- 
2.7.3

[PATCH v2 14/54] v4l: vsp1: bru: Don't program background color in control set handler

[Laurent Pinchart]

The datasheet clearly states that all but a few registers can't be
modified when the device is running. Programming the background color
in the control set handler is thus prohibited. Program it when starting
the module instead.

This requires storing the background color value internally as the
module can be started from the frame completion interrupt handler, and
accessing control values requires taking a mutex.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c | 15 +++++++++------
 drivers/media/platform/vsp1/vsp1_bru.h |  2 ++
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 565c8b2edf19..16345ec66870 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -42,13 +42,9 @@ static int bru_s_ctrl(struct v4l2_ctrl *ctrl)
 	struct vsp1_bru *bru =
 		container_of(ctrl->handler, struct vsp1_bru, ctrls);
 
-	if (!vsp1_entity_is_streaming(&bru->entity))
-		return 0;
-
 	switch (ctrl->id) {
 	case V4L2_CID_BG_COLOR:
-		vsp1_bru_write(bru, VI6_BRU_VIRRPF_COL, ctrl->val |
-			       (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
+		bru->bgcolor = ctrl->val;
 		break;
 	}
 
@@ -95,12 +91,17 @@ static int bru_s_stream(struct v4l2_subdev *subdev, int enable)
 		       flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
 		       0 : VI6_BRU_INCTRL_NRM);
 
-	/* Set the background position to cover the whole output image. */
+	/* Set the background position to cover the whole output image and
+	 * configure its color.
+	 */
 	vsp1_bru_write(bru, VI6_BRU_VIRRPF_SIZE,
 		       (format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
 		       (format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
 	vsp1_bru_write(bru, VI6_BRU_VIRRPF_LOC, 0);
 
+	vsp1_bru_write(bru, VI6_BRU_VIRRPF_COL, bru->bgcolor |
+		       (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
+
 	/* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
 	 * unit with a NOP operation to make BRU input 1 available as the
 	 * Blend/ROP unit B SRC input.
@@ -440,6 +441,8 @@ struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1)
 	v4l2_ctrl_new_std(&bru->ctrls, &bru_ctrl_ops, V4L2_CID_BG_COLOR,
 			  0, 0xffffff, 1, 0);
 
+	bru->bgcolor = 0;
+
 	bru->entity.subdev.ctrl_handler = &bru->ctrls;
 
 	if (bru->ctrls.error) {
diff --git a/drivers/media/platform/vsp1/vsp1_bru.h b/drivers/media/platform/vsp1/vsp1_bru.h
index dbac9686ea69..4e7d2e79b940 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.h
+++ b/drivers/media/platform/vsp1/vsp1_bru.h
@@ -33,6 +33,8 @@ struct vsp1_bru {
 		struct vsp1_rwpf *rpf;
 		struct v4l2_rect compose;
 	} inputs[VSP1_MAX_RPF];
+
+	u32 bgcolor;
 };
 
 static inline struct vsp1_bru *to_bru(struct v4l2_subdev *subdev)
-- 
2.7.3

[PATCH v2 15/54] v4l: vsp1: rwpf: Don't program alpha value in control set handler

[Laurent Pinchart]

The datasheet clearly states that all but a few registers can't be
modified when the device is running. Programming the alpha value in
the control set handler is thus prohibited. Program it when starting the
module instead.

This requires storing the alpha value internally as the module can be
started from the frame completion interrupt handler, and accessing
control values requires taking a mutex.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_rpf.c  | 51 +++---------------------------
 drivers/media/platform/vsp1/vsp1_rwpf.c | 35 +++++++++++++++++++++
 drivers/media/platform/vsp1/vsp1_rwpf.h |  5 ++-
 drivers/media/platform/vsp1/vsp1_wpf.c  | 55 ++-------------------------------
 4 files changed, 47 insertions(+), 99 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 7853e0f1d526..8721c82801ca 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -33,36 +33,6 @@ static inline void vsp1_rpf_write(struct vsp1_rwpf *rpf, u32 reg, u32 data)
 }
 
 /* -----------------------------------------------------------------------------
- * Controls
- */
-
-static int rpf_s_ctrl(struct v4l2_ctrl *ctrl)
-{
-	struct vsp1_rwpf *rpf =
-		container_of(ctrl->handler, struct vsp1_rwpf, ctrls);
-	struct vsp1_pipeline *pipe;
-
-	if (!vsp1_entity_is_streaming(&rpf->entity))
-		return 0;
-
-	switch (ctrl->id) {
-	case V4L2_CID_ALPHA_COMPONENT:
-		vsp1_rpf_write(rpf, VI6_RPF_VRTCOL_SET,
-			       ctrl->val << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
-
-		pipe = to_vsp1_pipeline(&rpf->entity.subdev.entity);
-		vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, ctrl->val);
-		break;
-	}
-
-	return 0;
-}
-
-static const struct v4l2_ctrl_ops rpf_ctrl_ops = {
-	.s_ctrl = rpf_s_ctrl,
-};
-
-/* -----------------------------------------------------------------------------
  * V4L2 Subdevice Core Operations
  */
 
@@ -70,7 +40,6 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 {
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
 	struct vsp1_rwpf *rpf = to_rwpf(subdev);
-	struct vsp1_device *vsp1 = rpf->entity.vsp1;
 	const struct vsp1_format_info *fmtinfo = rpf->fmtinfo;
 	const struct v4l2_pix_format_mplane *format = &rpf->format;
 	const struct v4l2_rect *crop = &rpf->crop;
@@ -151,13 +120,10 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 		       (fmtinfo->alpha ? VI6_RPF_ALPH_SEL_ASEL_PACKED
 				       : VI6_RPF_ALPH_SEL_ASEL_FIXED));
 
-	if (vsp1->info->uapi)
-		mutex_lock(rpf->ctrls.lock);
 	vsp1_rpf_write(rpf, VI6_RPF_VRTCOL_SET,
-		       rpf->alpha->cur.val << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
-	vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, rpf->alpha->cur.val);
-	if (vsp1->info->uapi)
-		mutex_unlock(rpf->ctrls.lock);
+		       rpf->alpha << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
+
+	vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, rpf->alpha);
 
 	vsp1_rpf_write(rpf, VI6_RPF_MSK_CTRL, 0);
 	vsp1_rpf_write(rpf, VI6_RPF_CKEY_CTRL, 0);
@@ -257,17 +223,10 @@ struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index)
 	vsp1_entity_init_formats(subdev, NULL);
 
 	/* Initialize the control handler. */
-	v4l2_ctrl_handler_init(&rpf->ctrls, 1);
-	rpf->alpha = v4l2_ctrl_new_std(&rpf->ctrls, &rpf_ctrl_ops,
-				       V4L2_CID_ALPHA_COMPONENT,
-				       0, 255, 1, 255);
-
-	rpf->entity.subdev.ctrl_handler = &rpf->ctrls;
-
-	if (rpf->ctrls.error) {
+	ret = vsp1_rwpf_init_ctrls(rpf);
+	if (ret < 0) {
 		dev_err(vsp1->dev, "rpf%u: failed to initialize controls\n",
 			index);
-		ret = rpf->ctrls.error;
 		goto error;
 	}
 
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 9688c219b30e..ba50386db35c 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -230,3 +230,38 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 
 	return 0;
 }
+
+/* -----------------------------------------------------------------------------
+ * Controls
+ */
+
+static int vsp1_rwpf_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+	struct vsp1_rwpf *rwpf =
+		container_of(ctrl->handler, struct vsp1_rwpf, ctrls);
+
+	switch (ctrl->id) {
+	case V4L2_CID_ALPHA_COMPONENT:
+		rwpf->alpha = ctrl->val;
+		break;
+	}
+
+	return 0;
+}
+
+static const struct v4l2_ctrl_ops vsp1_rwpf_ctrl_ops = {
+	.s_ctrl = vsp1_rwpf_s_ctrl,
+};
+
+int vsp1_rwpf_init_ctrls(struct vsp1_rwpf *rwpf)
+{
+	rwpf->alpha = 255;
+
+	v4l2_ctrl_handler_init(&rwpf->ctrls, 1);
+	v4l2_ctrl_new_std(&rwpf->ctrls, &vsp1_rwpf_ctrl_ops,
+			  V4L2_CID_ALPHA_COMPONENT, 0, 255, 1, 255);
+
+	rwpf->entity.subdev.ctrl_handler = &rwpf->ctrls;
+
+	return rwpf->ctrls.error;
+}
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index d04df39b2737..66af2a06dd8b 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -42,7 +42,6 @@ struct vsp1_rwpf_operations {
 struct vsp1_rwpf {
 	struct vsp1_entity entity;
 	struct v4l2_ctrl_handler ctrls;
-	struct v4l2_ctrl *alpha;
 
 	struct vsp1_video *video;
 
@@ -59,6 +58,8 @@ struct vsp1_rwpf {
 	} location;
 	struct v4l2_rect crop;
 
+	unsigned int alpha;
+
 	unsigned int offsets[2];
 	dma_addr_t buf_addr[3];
 
@@ -73,6 +74,8 @@ static inline struct vsp1_rwpf *to_rwpf(struct v4l2_subdev *subdev)
 struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index);
 struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index);
 
+int vsp1_rwpf_init_ctrls(struct vsp1_rwpf *rwpf);
+
 int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
 			     struct v4l2_subdev_pad_config *cfg,
 			     struct v4l2_subdev_mbus_code_enum *code);
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 3640989b3fd5..1ca08f4d67c2 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -27,12 +27,6 @@
  * Device Access
  */
 
-static inline u32 vsp1_wpf_read(struct vsp1_rwpf *wpf, u32 reg)
-{
-	return vsp1_read(wpf->entity.vsp1,
-			 reg + wpf->entity.index * VI6_WPF_OFFSET);
-}
-
 static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf, u32 reg, u32 data)
 {
 	vsp1_mod_write(&wpf->entity,
@@ -40,35 +34,6 @@ static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf, u32 reg, u32 data)
 }
 
 /* -----------------------------------------------------------------------------
- * Controls
- */
-
-static int wpf_s_ctrl(struct v4l2_ctrl *ctrl)
-{
-	struct vsp1_rwpf *wpf =
-		container_of(ctrl->handler, struct vsp1_rwpf, ctrls);
-	u32 value;
-
-	if (!vsp1_entity_is_streaming(&wpf->entity))
-		return 0;
-
-	switch (ctrl->id) {
-	case V4L2_CID_ALPHA_COMPONENT:
-		value = vsp1_wpf_read(wpf, VI6_WPF_OUTFMT);
-		value &= ~VI6_WPF_OUTFMT_PDV_MASK;
-		value |= ctrl->val << VI6_WPF_OUTFMT_PDV_SHIFT;
-		vsp1_wpf_write(wpf, VI6_WPF_OUTFMT, value);
-		break;
-	}
-
-	return 0;
-}
-
-static const struct v4l2_ctrl_ops wpf_ctrl_ops = {
-	.s_ctrl = wpf_s_ctrl,
-};
-
-/* -----------------------------------------------------------------------------
  * V4L2 Subdevice Core Operations
  */
 
@@ -153,15 +118,8 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	    wpf->entity.formats[RWPF_PAD_SOURCE].code)
 		outfmt |= VI6_WPF_OUTFMT_CSC;
 
-	/* Take the control handler lock to ensure that the PDV value won't be
-	 * changed behind our back by a set control operation.
-	 */
-	if (vsp1->info->uapi)
-		mutex_lock(wpf->ctrls.lock);
-	outfmt |= wpf->alpha->cur.val << VI6_WPF_OUTFMT_PDV_SHIFT;
+	outfmt |= wpf->alpha << VI6_WPF_OUTFMT_PDV_SHIFT;
 	vsp1_wpf_write(wpf, VI6_WPF_OUTFMT, outfmt);
-	if (vsp1->info->uapi)
-		mutex_unlock(wpf->ctrls.lock);
 
 	vsp1_mod_write(&wpf->entity, VI6_DPR_WPF_FPORCH(wpf->entity.index),
 		       VI6_DPR_WPF_FPORCH_FP_WPFN);
@@ -274,17 +232,10 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	vsp1_entity_init_formats(subdev, NULL);
 
 	/* Initialize the control handler. */
-	v4l2_ctrl_handler_init(&wpf->ctrls, 1);
-	wpf->alpha = v4l2_ctrl_new_std(&wpf->ctrls, &wpf_ctrl_ops,
-				       V4L2_CID_ALPHA_COMPONENT,
-				       0, 255, 1, 255);
-
-	wpf->entity.subdev.ctrl_handler = &wpf->ctrls;
-
-	if (wpf->ctrls.error) {
+	ret = vsp1_rwpf_init_ctrls(wpf);
+	if (ret < 0) {
 		dev_err(vsp1->dev, "wpf%u: failed to initialize controls\n",
 			index);
-		ret = wpf->ctrls.error;
 		goto error;
 	}
 
-- 
2.7.3

[PATCH v2 16/54] v4l: vsp1: sru: Don't program intensity in control set handler

[Laurent Pinchart]

The datasheet clearly states that all but a few registers can't be
modified when the device is running. Programming the intensity
parameters in the control set handler is thus prohibited. Program it
when starting the module instead.

This requires storing the intensity value internally as the module can
be started from the frame completion interrupt handler, and accessing
control values requires taking a mutex.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_sru.c | 35 +++++++++-------------------------
 drivers/media/platform/vsp1/vsp1_sru.h |  2 ++
 2 files changed, 11 insertions(+), 26 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 149ee1cd0b5a..d2c705563cd7 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -26,11 +26,6 @@
  * Device Access
  */
 
-static inline u32 vsp1_sru_read(struct vsp1_sru *sru, u32 reg)
-{
-	return vsp1_read(sru->entity.vsp1, reg);
-}
-
 static inline void vsp1_sru_write(struct vsp1_sru *sru, u32 reg, u32 data)
 {
 	vsp1_write(sru->entity.vsp1, reg, data);
@@ -82,20 +77,10 @@ static int sru_s_ctrl(struct v4l2_ctrl *ctrl)
 {
 	struct vsp1_sru *sru =
 		container_of(ctrl->handler, struct vsp1_sru, ctrls);
-	const struct vsp1_sru_param *param;
-	u32 value;
 
 	switch (ctrl->id) {
 	case V4L2_CID_VSP1_SRU_INTENSITY:
-		param = &vsp1_sru_params[ctrl->val - 1];
-
-		value = vsp1_sru_read(sru, VI6_SRU_CTRL0);
-		value &= ~(VI6_SRU_CTRL0_PARAM0_MASK |
-			   VI6_SRU_CTRL0_PARAM1_MASK);
-		value |= param->ctrl0;
-		vsp1_sru_write(sru, VI6_SRU_CTRL0, value);
-
-		vsp1_sru_write(sru, VI6_SRU_CTRL2, param->ctrl2);
+		sru->intensity = ctrl->val;
 		break;
 	}
 
@@ -123,6 +108,7 @@ static const struct v4l2_ctrl_config sru_intensity_control = {
 
 static int sru_s_stream(struct v4l2_subdev *subdev, int enable)
 {
+	const struct vsp1_sru_param *param;
 	struct vsp1_sru *sru = to_sru(subdev);
 	struct v4l2_mbus_framefmt *input;
 	struct v4l2_mbus_framefmt *output;
@@ -148,18 +134,13 @@ static int sru_s_stream(struct v4l2_subdev *subdev, int enable)
 	if (input->width != output->width)
 		ctrl0 |= VI6_SRU_CTRL0_MODE_UPSCALE;
 
-	/* Take the control handler lock to ensure that the CTRL0 value won't be
-	 * changed behind our back by a set control operation.
-	 */
-	if (sru->entity.vsp1->info->uapi)
-		mutex_lock(sru->ctrls.lock);
-	ctrl0 |= vsp1_sru_read(sru, VI6_SRU_CTRL0)
-	       & (VI6_SRU_CTRL0_PARAM0_MASK | VI6_SRU_CTRL0_PARAM1_MASK);
-	vsp1_sru_write(sru, VI6_SRU_CTRL0, ctrl0);
-	if (sru->entity.vsp1->info->uapi)
-		mutex_unlock(sru->ctrls.lock);
+	param = &vsp1_sru_params[sru->intensity - 1];
+
+	ctrl0 |= param->ctrl0;
 
+	vsp1_sru_write(sru, VI6_SRU_CTRL0, ctrl0);
 	vsp1_sru_write(sru, VI6_SRU_CTRL1, VI6_SRU_CTRL1_PARAM5);
+	vsp1_sru_write(sru, VI6_SRU_CTRL2, param->ctrl2);
 
 	return 0;
 }
@@ -378,6 +359,8 @@ struct vsp1_sru *vsp1_sru_create(struct vsp1_device *vsp1)
 	v4l2_ctrl_handler_init(&sru->ctrls, 1);
 	v4l2_ctrl_new_custom(&sru->ctrls, &sru_intensity_control, NULL);
 
+	sru->intensity = 1;
+
 	sru->entity.subdev.ctrl_handler = &sru->ctrls;
 
 	if (sru->ctrls.error) {
diff --git a/drivers/media/platform/vsp1/vsp1_sru.h b/drivers/media/platform/vsp1/vsp1_sru.h
index b6768bf3dc47..85e241457af2 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.h
+++ b/drivers/media/platform/vsp1/vsp1_sru.h
@@ -28,6 +28,8 @@ struct vsp1_sru {
 	struct vsp1_entity entity;
 
 	struct v4l2_ctrl_handler ctrls;
+
+	unsigned int intensity;
 };
 
 static inline struct vsp1_sru *to_sru(struct v4l2_subdev *subdev)
-- 
2.7.3

[PATCH v2 17/54] v4l: vsp1: Don't setup control handler when starting streaming

[Laurent Pinchart]

The control handler set operations don't program the hardware anymore,
there's thus no need to call them when starting the stream.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    |  5 +----
 drivers/media/platform/vsp1/vsp1_entity.c | 18 +-----------------
 drivers/media/platform/vsp1/vsp1_entity.h |  2 +-
 drivers/media/platform/vsp1/vsp1_rpf.c    |  5 +----
 drivers/media/platform/vsp1/vsp1_sru.c    |  5 +----
 drivers/media/platform/vsp1/vsp1_wpf.c    |  5 +----
 6 files changed, 6 insertions(+), 34 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 16345ec66870..5feec203e6fb 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -66,11 +66,8 @@ static int bru_s_stream(struct v4l2_subdev *subdev, int enable)
 	struct v4l2_mbus_framefmt *format;
 	unsigned int flags;
 	unsigned int i;
-	int ret;
 
-	ret = vsp1_entity_set_streaming(&bru->entity, enable);
-	if (ret < 0)
-		return ret;
+	vsp1_entity_set_streaming(&bru->entity, enable);
 
 	if (!enable)
 		return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index a94f544dcc77..6b425ae9aba3 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -45,29 +45,13 @@ bool vsp1_entity_is_streaming(struct vsp1_entity *entity)
 	return streaming;
 }
 
-int vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming)
+void vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming)
 {
 	unsigned long flags;
-	int ret;
 
 	spin_lock_irqsave(&entity->lock, flags);
 	entity->streaming = streaming;
 	spin_unlock_irqrestore(&entity->lock, flags);
-
-	if (!streaming)
-		return 0;
-
-	if (!entity->vsp1->info->uapi || !entity->subdev.ctrl_handler)
-		return 0;
-
-	ret = v4l2_ctrl_handler_setup(entity->subdev.ctrl_handler);
-	if (ret < 0) {
-		spin_lock_irqsave(&entity->lock, flags);
-		entity->streaming = false;
-		spin_unlock_irqrestore(&entity->lock, flags);
-	}
-
-	return ret;
 }
 
 void vsp1_entity_route_setup(struct vsp1_entity *source)
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 259880e524fe..c0d6db82ebfb 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -101,7 +101,7 @@ void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_pad_config *cfg);
 
 bool vsp1_entity_is_streaming(struct vsp1_entity *entity);
-int vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming);
+void vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming);
 
 void vsp1_entity_route_setup(struct vsp1_entity *source);
 
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 8721c82801ca..8c7c385ec046 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -45,11 +45,8 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	const struct v4l2_rect *crop = &rpf->crop;
 	u32 pstride;
 	u32 infmt;
-	int ret;
 
-	ret = vsp1_entity_set_streaming(&rpf->entity, enable);
-	if (ret < 0)
-		return ret;
+	vsp1_entity_set_streaming(&rpf->entity, enable);
 
 	if (!enable)
 		return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index d2c705563cd7..a97541492af8 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -113,11 +113,8 @@ static int sru_s_stream(struct v4l2_subdev *subdev, int enable)
 	struct v4l2_mbus_framefmt *input;
 	struct v4l2_mbus_framefmt *output;
 	u32 ctrl0;
-	int ret;
 
-	ret = vsp1_entity_set_streaming(&sru->entity, enable);
-	if (ret < 0)
-		return ret;
+	vsp1_entity_set_streaming(&sru->entity, enable);
 
 	if (!enable)
 		return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 1ca08f4d67c2..a7101f700d9e 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -46,11 +46,8 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	unsigned int i;
 	u32 srcrpf = 0;
 	u32 outfmt = 0;
-	int ret;
 
-	ret = vsp1_entity_set_streaming(&wpf->entity, enable);
-	if (ret < 0)
-		return ret;
+	vsp1_entity_set_streaming(&wpf->entity, enable);
 
 	if (!enable) {
 		vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index), 0);
-- 
2.7.3

[PATCH v2 18/54] v4l: vsp1: Enable display list support for the HS[IT], LUT, SRU and UDS

[Laurent Pinchart]

Those modules were left out of display list integration as they're not
used by the DRM pipeline. To prepare for display list support in non-DRM
pipelines use the module write API to set registers.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_hsit.c | 2 +-
 drivers/media/platform/vsp1/vsp1_lut.c  | 2 +-
 drivers/media/platform/vsp1/vsp1_sru.c  | 2 +-
 drivers/media/platform/vsp1/vsp1_uds.c  | 4 ++--
 4 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index ce42ce2e4847..e971dfa9714d 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -28,7 +28,7 @@
 
 static inline void vsp1_hsit_write(struct vsp1_hsit *hsit, u32 reg, u32 data)
 {
-	vsp1_write(hsit->entity.vsp1, reg, data);
+	vsp1_mod_write(&hsit->entity, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index f0cd4f79fbff..c24712fe5f2c 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -29,7 +29,7 @@
 
 static inline void vsp1_lut_write(struct vsp1_lut *lut, u32 reg, u32 data)
 {
-	vsp1_write(lut->entity.vsp1, reg, data);
+	vsp1_mod_write(&lut->entity, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index a97541492af8..7de62be37cff 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -28,7 +28,7 @@
 
 static inline void vsp1_sru_write(struct vsp1_sru *sru, u32 reg, u32 data)
 {
-	vsp1_write(sru->entity.vsp1, reg, data);
+	vsp1_mod_write(&sru->entity, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index b1881a0a314f..7eaf42a2b036 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -31,8 +31,8 @@
 
 static inline void vsp1_uds_write(struct vsp1_uds *uds, u32 reg, u32 data)
 {
-	vsp1_write(uds->entity.vsp1,
-		   reg + uds->entity.index * VI6_UDS_OFFSET, data);
+	vsp1_mod_write(&uds->entity, reg + uds->entity.index * VI6_UDS_OFFSET,
+		       data);
 }
 
 /* -----------------------------------------------------------------------------
-- 
2.7.3

[PATCH v2 19/54] v4l: vsp1: Don't configure RPF memory buffers before calculating offsets

[Laurent Pinchart]

The RPF source memory pointers need to be offset to take the crop
rectangle into account. Offsets are computed in the RPF stream start,
which can happen (when using the DRM pipeline) after calling the RPF
.set_memory() operation that programs the buffer addresses.

The .set_memory() operation tries to guard against the problem by
skipping programming of the registers when the module isn't streaming.
This will however only protect the first use of an RPF in a DRM
pipeline, as in all subsequent uses the module streaming flag will be
set and the .set_memory() operation will use potentially incorrect
offsets.

Fix this by allowing the caller to decide whether to program the
hardware immediately or just cache the addresses. While at it refactor
the memory set code and create a new vsp1_rwpf_set_memory() that cache
addresses and calls the .set_memory() operation to apply them to the
hardware.

As a side effect the driver now writes all three DMA address registers
regardless of the number of planes, and initializes unused addresses to
zero.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_drm.c   |  7 ++++--
 drivers/media/platform/vsp1/vsp1_rpf.c   | 37 ++++++++++----------------------
 drivers/media/platform/vsp1/vsp1_rwpf.c  | 26 ++++++++++++++++++++++
 drivers/media/platform/vsp1/vsp1_rwpf.h  | 11 ++++++++--
 drivers/media/platform/vsp1/vsp1_video.c |  9 ++++++--
 drivers/media/platform/vsp1/vsp1_wpf.c   | 10 ++++-----
 6 files changed, 62 insertions(+), 38 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 22f67360b750..96ad8072d5d6 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -420,12 +420,15 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 	rpf->location.left = dst->left;
 	rpf->location.top = dst->top;
 
-	/* Set the memory buffer address. */
+	/* Set the memory buffer address but don't apply the values to the
+	 * hardware as the crop offsets haven't been computed yet.
+	 */
 	memory.num_planes = fmtinfo->planes;
 	memory.addr[0] = mem[0];
 	memory.addr[1] = mem[1];
+	memory.addr[2] = 0;
 
-	rpf->ops->set_memory(rpf, &memory);
+	vsp1_rwpf_set_memory(rpf, &memory, false);
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 8c7c385ec046..3509202f1b4a 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -69,25 +69,20 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	pstride = format->plane_fmt[0].bytesperline
 		<< VI6_RPF_SRCM_PSTRIDE_Y_SHIFT;
 
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
-		       rpf->buf_addr[0] + rpf->offsets[0]);
-
 	if (format->num_planes > 1) {
 		rpf->offsets[1] = crop->top * format->plane_fmt[1].bytesperline
 				+ crop->left * fmtinfo->bpp[1] / 8;
 		pstride |= format->plane_fmt[1].bytesperline
 			<< VI6_RPF_SRCM_PSTRIDE_C_SHIFT;
-
-		vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
-			       rpf->buf_addr[1] + rpf->offsets[1]);
-
-		if (format->num_planes > 2)
-			vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
-				       rpf->buf_addr[2] + rpf->offsets[1]);
+	} else {
+		rpf->offsets[1] = 0;
 	}
 
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_PSTRIDE, pstride);
 
+	/* Now that the offsets have been computed program the DMA addresses. */
+	rpf->ops->set_memory(rpf);
+
 	/* Format */
 	infmt = VI6_RPF_INFMT_CIPM
 	      | (fmtinfo->hwfmt << VI6_RPF_INFMT_RDFMT_SHIFT);
@@ -154,24 +149,14 @@ static struct v4l2_subdev_ops rpf_ops = {
  * Video Device Operations
  */
 
-static void rpf_set_memory(struct vsp1_rwpf *rpf, struct vsp1_rwpf_memory *mem)
+static void rpf_set_memory(struct vsp1_rwpf *rpf)
 {
-	unsigned int i;
-
-	for (i = 0; i < 3; ++i)
-		rpf->buf_addr[i] = mem->addr[i];
-
-	if (!vsp1_entity_is_streaming(&rpf->entity))
-		return;
-
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
-		       mem->addr[0] + rpf->offsets[0]);
-	if (mem->num_planes > 1)
-		vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
-			       mem->addr[1] + rpf->offsets[1]);
-	if (mem->num_planes > 2)
-		vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
-			       mem->addr[2] + rpf->offsets[1]);
+		       rpf->buf_addr[0] + rpf->offsets[0]);
+	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
+		       rpf->buf_addr[1] + rpf->offsets[1]);
+	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
+		       rpf->buf_addr[2] + rpf->offsets[1]);
 }
 
 static const struct vsp1_rwpf_operations rpf_vdev_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index ba50386db35c..54070ccdc2ff 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -265,3 +265,29 @@ int vsp1_rwpf_init_ctrls(struct vsp1_rwpf *rwpf)
 
 	return rwpf->ctrls.error;
 }
+
+/* -----------------------------------------------------------------------------
+ * Buffers
+ */
+
+/**
+ * vsp1_rwpf_set_memory - Configure DMA addresses for a [RW]PF
+ * @rwpf: the [RW]PF instance
+ * @mem: DMA memory addresses
+ * @apply: whether to apply the configuration to the hardware
+ *
+ * This function stores the DMA addresses for all planes in the rwpf instance
+ * and optionally applies the configuration to hardware registers if the apply
+ * argument is set to true.
+ */
+void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf, struct vsp1_rwpf_memory *mem,
+			  bool apply)
+{
+	unsigned int i;
+
+	for (i = 0; i < 3; ++i)
+		rwpf->buf_addr[i] = mem->addr[i];
+
+	if (apply)
+		rwpf->ops->set_memory(rwpf);
+}
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 66af2a06dd8b..bda0416dc7db 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -34,9 +34,13 @@ struct vsp1_rwpf_memory {
 	unsigned int length[3];
 };
 
+/**
+ * struct vsp1_rwpf_operations - RPF and WPF operations
+ * @set_memory: Setup memory buffer access. This operation applies the settings
+ *		stored in the rwpf buf_addr field to the hardware.
+ */
 struct vsp1_rwpf_operations {
-	void (*set_memory)(struct vsp1_rwpf *rwpf,
-			   struct vsp1_rwpf_memory *mem);
+	void (*set_memory)(struct vsp1_rwpf *rwpf);
 };
 
 struct vsp1_rwpf {
@@ -93,4 +97,7 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 			    struct v4l2_subdev_pad_config *cfg,
 			    struct v4l2_subdev_selection *sel);
 
+void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf, struct vsp1_rwpf_memory *mem,
+			  bool apply);
+
 #endif /* __VSP1_RWPF_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index b97bbdb1a256..96b04fcd33ae 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -443,7 +443,7 @@ static void vsp1_video_frame_end(struct vsp1_pipeline *pipe,
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
-	video->rwpf->ops->set_memory(video->rwpf, &buf->mem);
+	vsp1_rwpf_set_memory(video->rwpf, &buf->mem, true);
 	pipe->buffers_ready |= 1 << video->pipe_index;
 
 	spin_unlock_irqrestore(&pipe->irqlock, flags);
@@ -522,6 +522,11 @@ static int vsp1_video_buffer_prepare(struct vb2_buffer *vb)
 			return -EINVAL;
 	}
 
+	for ( ; i < 3; ++i) {
+		buf->mem.addr[i] = 0;
+		buf->mem.length[i] = 0;
+	}
+
 	return 0;
 }
 
@@ -544,7 +549,7 @@ static void vsp1_video_buffer_queue(struct vb2_buffer *vb)
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
-	video->rwpf->ops->set_memory(video->rwpf, &buf->mem);
+	vsp1_rwpf_set_memory(video->rwpf, &buf->mem, true);
 	pipe->buffers_ready |= 1 << video->pipe_index;
 
 	if (vb2_is_streaming(&video->queue) &&
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index a7101f700d9e..978dd6cda9d3 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -157,13 +157,11 @@ static struct v4l2_subdev_ops wpf_ops = {
  * Video Device Operations
  */
 
-static void wpf_set_memory(struct vsp1_rwpf *wpf, struct vsp1_rwpf_memory *mem)
+static void wpf_set_memory(struct vsp1_rwpf *wpf)
 {
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, mem->addr[0]);
-	if (mem->num_planes > 1)
-		vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, mem->addr[1]);
-	if (mem->num_planes > 2)
-		vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, mem->addr[2]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, wpf->buf_addr[0]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, wpf->buf_addr[1]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, wpf->buf_addr[2]);
 }
 
 static const struct vsp1_rwpf_operations wpf_vdev_ops = {
-- 
2.7.3

[PATCH v2 20/54] v4l: vsp1: Remove unneeded entity streaming flag

[Laurent Pinchart]

The flag is set but never read, remove it.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    |  2 --
 drivers/media/platform/vsp1/vsp1_entity.c | 23 -----------------------
 drivers/media/platform/vsp1/vsp1_entity.h |  6 ------
 drivers/media/platform/vsp1/vsp1_rpf.c    |  2 --
 drivers/media/platform/vsp1/vsp1_sru.c    |  2 --
 drivers/media/platform/vsp1/vsp1_wpf.c    |  2 --
 6 files changed, 37 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 5feec203e6fb..1814d8dc1cd2 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -67,8 +67,6 @@ static int bru_s_stream(struct v4l2_subdev *subdev, int enable)
 	unsigned int flags;
 	unsigned int i;
 
-	vsp1_entity_set_streaming(&bru->entity, enable);
-
 	if (!enable)
 		return 0;
 
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 6b425ae9aba3..be67727f6f78 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -33,27 +33,6 @@ void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
 		vsp1_write(e->vsp1, reg, data);
 }
 
-bool vsp1_entity_is_streaming(struct vsp1_entity *entity)
-{
-	unsigned long flags;
-	bool streaming;
-
-	spin_lock_irqsave(&entity->lock, flags);
-	streaming = entity->streaming;
-	spin_unlock_irqrestore(&entity->lock, flags);
-
-	return streaming;
-}
-
-void vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming)
-{
-	unsigned long flags;
-
-	spin_lock_irqsave(&entity->lock, flags);
-	entity->streaming = streaming;
-	spin_unlock_irqrestore(&entity->lock, flags);
-}
-
 void vsp1_entity_route_setup(struct vsp1_entity *source)
 {
 	struct vsp1_entity *sink;
@@ -198,8 +177,6 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 	if (i == ARRAY_SIZE(vsp1_routes))
 		return -EINVAL;
 
-	spin_lock_init(&entity->lock);
-
 	entity->vsp1 = vsp1;
 	entity->source_pad = num_pads - 1;
 
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index c0d6db82ebfb..203872164f8e 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -73,9 +73,6 @@ struct vsp1_entity {
 
 	struct v4l2_subdev subdev;
 	struct v4l2_mbus_framefmt *formats;
-
-	spinlock_t lock;		/* Protects the streaming field */
-	bool streaming;
 };
 
 static inline struct vsp1_entity *to_vsp1_entity(struct v4l2_subdev *subdev)
@@ -100,9 +97,6 @@ vsp1_entity_get_pad_format(struct vsp1_entity *entity,
 void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_pad_config *cfg);
 
-bool vsp1_entity_is_streaming(struct vsp1_entity *entity);
-void vsp1_entity_set_streaming(struct vsp1_entity *entity, bool streaming);
-
 void vsp1_entity_route_setup(struct vsp1_entity *source);
 
 void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data);
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 3509202f1b4a..9857d633f61e 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -46,8 +46,6 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	u32 pstride;
 	u32 infmt;
 
-	vsp1_entity_set_streaming(&rpf->entity, enable);
-
 	if (!enable)
 		return 0;
 
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 7de62be37cff..5a7ffbd18043 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -114,8 +114,6 @@ static int sru_s_stream(struct v4l2_subdev *subdev, int enable)
 	struct v4l2_mbus_framefmt *output;
 	u32 ctrl0;
 
-	vsp1_entity_set_streaming(&sru->entity, enable);
-
 	if (!enable)
 		return 0;
 
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 978dd6cda9d3..cf18183370f4 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -47,8 +47,6 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	u32 srcrpf = 0;
 	u32 outfmt = 0;
 
-	vsp1_entity_set_streaming(&wpf->entity, enable);
-
 	if (!enable) {
 		vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index), 0);
 		vsp1_write(vsp1, wpf->entity.index * VI6_WPF_OFFSET +
-- 
2.7.3

[PATCH v2 21/54] v4l: vsp1: Document calling context of vsp1_pipeline_propagate_alpha()

[Laurent Pinchart]

The function can only be called from a s_stream handler as it requires a
valid display list context (due to calling vsp1_uds_set_alpha() which
writes to module registers). Document the requirement.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_pipe.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index cb67b8f80635..a9a754e17e8d 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -318,6 +318,9 @@ done:
  * to be scaled, we disable alpha scaling when the UDS input has a fixed alpha
  * value. The UDS then outputs a fixed alpha value which needs to be programmed
  * from the input RPF alpha.
+ *
+ * This function can only be called from a subdev s_stream handler as it
+ * requires a valid display list context.
  */
 void vsp1_pipeline_propagate_alpha(struct vsp1_pipeline *pipe,
 				   struct vsp1_entity *input,
-- 
2.7.3

[PATCH v2 22/54] v4l: vsp1: Fix 80 characters per line violations

[Laurent Pinchart]

Commit f7234138f14c ("v4l2-subdev: replace v4l2_subdev_fh by
v4l2_subdev_pad_config") introduced lots of 80 characters per line
violations. Fix them.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c  | 12 ++++++++----
 drivers/media/platform/vsp1/vsp1_lif.c  |  6 ++++--
 drivers/media/platform/vsp1/vsp1_lut.c  |  6 ++++--
 drivers/media/platform/vsp1/vsp1_rwpf.c | 12 ++++++++----
 drivers/media/platform/vsp1/vsp1_rwpf.h |  6 ++++--
 drivers/media/platform/vsp1/vsp1_sru.c  |  9 ++++++---
 drivers/media/platform/vsp1/vsp1_uds.c  |  9 ++++++---
 7 files changed, 40 insertions(+), 20 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 1814d8dc1cd2..fe6ebfa83311 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -195,7 +195,8 @@ static int bru_enum_mbus_code(struct v4l2_subdev *subdev,
 			return -EINVAL;
 
 		format = vsp1_entity_get_pad_format(&bru->entity, cfg,
-						    BRU_PAD_SINK(0), code->which);
+						    BRU_PAD_SINK(0),
+						    code->which);
 		code->code = format->code;
 	}
 
@@ -235,7 +236,8 @@ static struct v4l2_rect *bru_get_compose(struct vsp1_bru *bru,
 	}
 }
 
-static int bru_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int bru_get_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_bru *bru = to_bru(subdev);
@@ -246,7 +248,8 @@ static int bru_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_con
 	return 0;
 }
 
-static void bru_try_format(struct vsp1_bru *bru, struct v4l2_subdev_pad_config *cfg,
+static void bru_try_format(struct vsp1_bru *bru,
+			   struct v4l2_subdev_pad_config *cfg,
 			   unsigned int pad, struct v4l2_mbus_framefmt *fmt,
 			   enum v4l2_subdev_format_whence which)
 {
@@ -274,7 +277,8 @@ static void bru_try_format(struct vsp1_bru *bru, struct v4l2_subdev_pad_config *
 	fmt->colorspace = V4L2_COLORSPACE_SRGB;
 }
 
-static int bru_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int bru_set_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_bru *bru = to_bru(subdev);
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 56054fddb675..53725769442d 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -128,7 +128,8 @@ static int lif_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int lif_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int lif_get_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lif *lif = to_lif(subdev);
@@ -139,7 +140,8 @@ static int lif_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_con
 	return 0;
 }
 
-static int lif_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int lif_set_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lif *lif = to_lif(subdev);
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index c24712fe5f2c..1c0eefeee2f3 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -139,7 +139,8 @@ static int lut_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int lut_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int lut_get_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lut *lut = to_lut(subdev);
@@ -150,7 +151,8 @@ static int lut_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_con
 	return 0;
 }
 
-static int lut_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int lut_set_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lut *lut = to_lut(subdev);
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 54070ccdc2ff..0924079b920c 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -73,11 +73,13 @@ int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 }
 
 static struct v4l2_rect *
-vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf, struct v4l2_subdev_pad_config *cfg, u32 which)
+vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf, struct v4l2_subdev_pad_config *cfg,
+		   u32 which)
 {
 	switch (which) {
 	case V4L2_SUBDEV_FORMAT_TRY:
-		return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, cfg, RWPF_PAD_SINK);
+		return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, cfg,
+						RWPF_PAD_SINK);
 	case V4L2_SUBDEV_FORMAT_ACTIVE:
 		return &rwpf->crop;
 	default:
@@ -85,7 +87,8 @@ vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf, struct v4l2_subdev_pad_config *cfg, u
 	}
 }
 
-int vsp1_rwpf_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
+			 struct v4l2_subdev_pad_config *cfg,
 			 struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
@@ -96,7 +99,8 @@ int vsp1_rwpf_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_conf
 	return 0;
 }
 
-int vsp1_rwpf_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
+			 struct v4l2_subdev_pad_config *cfg,
 			 struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index bda0416dc7db..57f15d45f8bb 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -86,9 +86,11 @@ int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
 int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_pad_config *cfg,
 			      struct v4l2_subdev_frame_size_enum *fse);
-int vsp1_rwpf_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
+			 struct v4l2_subdev_pad_config *cfg,
 			 struct v4l2_subdev_format *fmt);
-int vsp1_rwpf_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
+			 struct v4l2_subdev_pad_config *cfg,
 			 struct v4l2_subdev_format *fmt);
 int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
 			    struct v4l2_subdev_pad_config *cfg,
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 5a7ffbd18043..d81a5b6300de 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -209,7 +209,8 @@ static int sru_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int sru_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int sru_get_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_sru *sru = to_sru(subdev);
@@ -220,7 +221,8 @@ static int sru_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_con
 	return 0;
 }
 
-static void sru_try_format(struct vsp1_sru *sru, struct v4l2_subdev_pad_config *cfg,
+static void sru_try_format(struct vsp1_sru *sru,
+			   struct v4l2_subdev_pad_config *cfg,
 			   unsigned int pad, struct v4l2_mbus_framefmt *fmt,
 			   enum v4l2_subdev_format_whence which)
 {
@@ -271,7 +273,8 @@ static void sru_try_format(struct vsp1_sru *sru, struct v4l2_subdev_pad_config *
 	fmt->colorspace = V4L2_COLORSPACE_SRGB;
 }
 
-static int sru_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int sru_set_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_sru *sru = to_sru(subdev);
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 7eaf42a2b036..286d0daea810 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -222,7 +222,8 @@ static int uds_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int uds_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int uds_get_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_uds *uds = to_uds(subdev);
@@ -233,7 +234,8 @@ static int uds_get_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_con
 	return 0;
 }
 
-static void uds_try_format(struct vsp1_uds *uds, struct v4l2_subdev_pad_config *cfg,
+static void uds_try_format(struct vsp1_uds *uds,
+			   struct v4l2_subdev_pad_config *cfg,
 			   unsigned int pad, struct v4l2_mbus_framefmt *fmt,
 			   enum v4l2_subdev_format_whence which)
 {
@@ -269,7 +271,8 @@ static void uds_try_format(struct vsp1_uds *uds, struct v4l2_subdev_pad_config *
 	fmt->colorspace = V4L2_COLORSPACE_SRGB;
 }
 
-static int uds_set_format(struct v4l2_subdev *subdev, struct v4l2_subdev_pad_config *cfg,
+static int uds_set_format(struct v4l2_subdev *subdev,
+			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_uds *uds = to_uds(subdev);
-- 
2.7.3

[PATCH v2 23/54] v4l: vsp1: Add header display list support

[Laurent Pinchart]

Display lists can operate in header or headerless mode. The headerless
mode is only available on WPF0, to be used with the display engine. All
other WPF instances can only use display lists in header mode.

Implement support for header mode to prepare for display list usage on
WPFs other than 0.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_dl.c  | 74 +++++++++++++++++++++++++++++++---
 drivers/media/platform/vsp1/vsp1_dl.h  |  1 +
 drivers/media/platform/vsp1/vsp1_wpf.c |  2 +-
 3 files changed, 71 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 0b2896c04f4f..596f6a67f1bb 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -28,9 +28,23 @@
  * - DL swap
  */
 
+#define VSP1_DL_HEADER_SIZE		76
 #define VSP1_DL_BODY_SIZE		(2 * 4 * 256)
 #define VSP1_DL_NUM_LISTS		3
 
+#define VSP1_DLH_INT_ENABLE		(1 << 1)
+#define VSP1_DLH_AUTO_START		(1 << 0)
+
+struct vsp1_dl_header {
+	u32 num_lists;
+	struct {
+		u32 num_bytes;
+		u32 addr;
+	} lists[8];
+	u32 next_header;
+	u32 flags;
+} __attribute__((__packed__));
+
 struct vsp1_dl_entry {
 	u32 addr;
 	u32 data;
@@ -41,6 +55,7 @@ struct vsp1_dl_list {
 
 	struct vsp1_dl_manager *dlm;
 
+	struct vsp1_dl_header *header;
 	struct vsp1_dl_entry *body;
 	dma_addr_t dma;
 	size_t size;
@@ -48,8 +63,15 @@ struct vsp1_dl_list {
 	int reg_count;
 };
 
+enum vsp1_dl_mode {
+	VSP1_DL_MODE_HEADER,
+	VSP1_DL_MODE_HEADERLESS,
+};
+
 /**
  * struct vsp1_dl_manager - Display List manager
+ * @index: index of the related WPF
+ * @mode: display list operation mode (header or headerless)
  * @vsp1: the VSP1 device
  * @lock: protects the active, queued and pending lists
  * @free: array of all free display lists
@@ -58,6 +80,8 @@ struct vsp1_dl_list {
  * @pending: list waiting to be queued to the hardware
  */
 struct vsp1_dl_manager {
+	unsigned int index;
+	enum vsp1_dl_mode mode;
 	struct vsp1_device *vsp1;
 
 	spinlock_t lock;
@@ -74,27 +98,44 @@ struct vsp1_dl_manager {
 static struct vsp1_dl_list *vsp1_dl_list_alloc(struct vsp1_dl_manager *dlm)
 {
 	struct vsp1_dl_list *dl;
+	size_t header_size;
+
+	/* The body needs to be aligned on a 8 bytes boundary, pad the header
+	 * size to allow allocating both in a single operation.
+	 */
+	header_size = dlm->mode == VSP1_DL_MODE_HEADER
+		    ? ALIGN(sizeof(struct vsp1_dl_header), 8)
+		    : 0;
 
 	dl = kzalloc(sizeof(*dl), GFP_KERNEL);
 	if (!dl)
 		return NULL;
 
 	dl->dlm = dlm;
-	dl->size = VSP1_DL_BODY_SIZE;
+	dl->size = header_size + VSP1_DL_BODY_SIZE;
 
-	dl->body = dma_alloc_writecombine(dlm->vsp1->dev, dl->size, &dl->dma,
-					  GFP_KERNEL);
-	if (!dl->body) {
+	dl->header = dma_alloc_writecombine(dlm->vsp1->dev, dl->size, &dl->dma,
+					    GFP_KERNEL);
+	if (!dl->header) {
 		kfree(dl);
 		return NULL;
 	}
 
+	if (dlm->mode == VSP1_DL_MODE_HEADER) {
+		memset(dl->header, 0, sizeof(*dl->header));
+		dl->header->lists[0].addr = dl->dma + header_size;
+		dl->header->flags = VSP1_DLH_INT_ENABLE;
+	}
+
+	dl->body = ((void *)dl->header) + header_size;
+
 	return dl;
 }
 
 static void vsp1_dl_list_free(struct vsp1_dl_list *dl)
 {
-	dma_free_writecombine(dl->dlm->vsp1->dev, dl->size, dl->body, dl->dma);
+	dma_free_writecombine(dl->dlm->vsp1->dev, dl->size, dl->header,
+			      dl->dma);
 	kfree(dl);
 }
 
@@ -160,6 +201,18 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
 
 	spin_lock_irqsave(&dlm->lock, flags);
 
+	if (dl->dlm->mode == VSP1_DL_MODE_HEADER) {
+		/* Program the hardware with the display list body address and
+		 * size. In header mode the caller guarantees that the hardware
+		 * is idle at this point.
+		 */
+		dl->header->lists[0].num_bytes = dl->reg_count * 8;
+		vsp1_write(vsp1, VI6_DL_HDR_ADDR(dlm->index), dl->dma);
+
+		dlm->active = dl;
+		goto done;
+	}
+
 	/* Once the UPD bit has been set the hardware can start processing the
 	 * display list at any time and we can't touch the address and size
 	 * registers. In that case mark the update as pending, it will be
@@ -215,6 +268,13 @@ void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
 	vsp1_dl_list_put(dlm->active);
 	dlm->active = NULL;
 
+	/* Header mode is used for mem-to-mem pipelines only. We don't need to
+	 * perform any operation as there can't be any new display list queued
+	 * in that case.
+	 */
+	if (dlm->mode == VSP1_DL_MODE_HEADER)
+		goto done;
+
 	/* The UPD bit set indicates that the commit operation raced with the
 	 * interrupt and occurred after the frame end event and UPD clear but
 	 * before interrupt processing. The hardware hasn't taken the update
@@ -277,6 +337,7 @@ void vsp1_dlm_reset(struct vsp1_dl_manager *dlm)
 }
 
 struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
+					unsigned int index,
 					unsigned int prealloc)
 {
 	struct vsp1_dl_manager *dlm;
@@ -286,6 +347,9 @@ struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
 	if (!dlm)
 		return NULL;
 
+	dlm->index = index;
+	dlm->mode = index == 0 && !vsp1->info->uapi
+		  ? VSP1_DL_MODE_HEADERLESS : VSP1_DL_MODE_HEADER;
 	dlm->vsp1 = vsp1;
 
 	spin_lock_init(&dlm->lock);
diff --git a/drivers/media/platform/vsp1/vsp1_dl.h b/drivers/media/platform/vsp1/vsp1_dl.h
index 46f7ae337374..571ed6d8e7c2 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.h
+++ b/drivers/media/platform/vsp1/vsp1_dl.h
@@ -22,6 +22,7 @@ struct vsp1_dl_manager;
 void vsp1_dlm_setup(struct vsp1_device *vsp1);
 
 struct vsp1_dl_manager *vsp1_dlm_create(struct vsp1_device *vsp1,
+					unsigned int index,
 					unsigned int prealloc);
 void vsp1_dlm_destroy(struct vsp1_dl_manager *dlm);
 void vsp1_dlm_reset(struct vsp1_dl_manager *dlm);
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index cf18183370f4..d82502681bc3 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -202,7 +202,7 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 
 	/* Initialize the display list manager if the WPF is used for display */
 	if ((vsp1->info->features & VSP1_HAS_LIF) && index == 0) {
-		wpf->dlm = vsp1_dlm_create(vsp1, 4);
+		wpf->dlm = vsp1_dlm_create(vsp1, index, 4);
 		if (!wpf->dlm) {
 			ret = -ENOMEM;
 			goto error;
-- 
2.7.3

[PATCH v2 24/54] v4l: vsp1: Use display lists with the userspace API

[Laurent Pinchart]

Don't restrict display list usage to the DRM pipeline, use them
unconditionally. This prepares the driver to support the request API.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_dl.c     |  11 +--
 drivers/media/platform/vsp1/vsp1_drm.c    |  23 ++---
 drivers/media/platform/vsp1/vsp1_entity.c |   5 +-
 drivers/media/platform/vsp1/vsp1_pipe.c   |  33 +------
 drivers/media/platform/vsp1/vsp1_rpf.c    |   9 +-
 drivers/media/platform/vsp1/vsp1_rwpf.c   |  26 ------
 drivers/media/platform/vsp1/vsp1_rwpf.h   |  18 ++--
 drivers/media/platform/vsp1/vsp1_video.c  | 145 +++++++++++++++++++++---------
 drivers/media/platform/vsp1/vsp1_wpf.c    |  18 ++--
 9 files changed, 142 insertions(+), 146 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 596f6a67f1bb..5a24d494059b 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -312,14 +312,15 @@ done:
 /* Hardware Setup */
 void vsp1_dlm_setup(struct vsp1_device *vsp1)
 {
-	u32 ctrl = (256 << VI6_DL_CTRL_AR_WAIT_SHIFT);
+	u32 ctrl = (256 << VI6_DL_CTRL_AR_WAIT_SHIFT)
+		 | VI6_DL_CTRL_DC2 | VI6_DL_CTRL_DC1 | VI6_DL_CTRL_DC0
+		 | VI6_DL_CTRL_DLE;
 
-	/* The DRM pipeline operates with header-less display lists in
-	 * Continuous Frame Mode.
+	/* The DRM pipeline operates with display lists in Continuous Frame
+	 * Mode, all other pipelines use manual start.
 	 */
 	if (vsp1->drm)
-		ctrl |= VI6_DL_CTRL_DC2 | VI6_DL_CTRL_DC1 | VI6_DL_CTRL_DC0
-		     |  VI6_DL_CTRL_DLE | VI6_DL_CTRL_CFM0 | VI6_DL_CTRL_NH0;
+		ctrl |= VI6_DL_CTRL_CFM0 | VI6_DL_CTRL_NH0;
 
 	vsp1_write(vsp1, VI6_DL_CTRL, ctrl);
 	vsp1_write(vsp1, VI6_DL_SWAP, VI6_DL_SWAP_LWS);
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 96ad8072d5d6..a73018c9e8b5 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -36,11 +36,6 @@ void vsp1_drm_display_start(struct vsp1_device *vsp1)
 	vsp1_dlm_irq_display_start(vsp1->drm->pipe.output->dlm);
 }
 
-void vsp1_drm_frame_end(struct vsp1_pipeline *pipe)
-{
-	vsp1_dlm_irq_frame_end(pipe->output->dlm);
-}
-
 /* -----------------------------------------------------------------------------
  * DU Driver API
  */
@@ -280,7 +275,6 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 	const struct vsp1_format_info *fmtinfo;
 	struct v4l2_subdev_selection sel;
 	struct v4l2_subdev_format format;
-	struct vsp1_rwpf_memory memory;
 	struct vsp1_rwpf *rpf;
 	unsigned long flags;
 	int ret;
@@ -420,15 +414,12 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 	rpf->location.left = dst->left;
 	rpf->location.top = dst->top;
 
-	/* Set the memory buffer address but don't apply the values to the
+	/* Cache the memory buffer address but don't apply the values to the
 	 * hardware as the crop offsets haven't been computed yet.
 	 */
-	memory.num_planes = fmtinfo->planes;
-	memory.addr[0] = mem[0];
-	memory.addr[1] = mem[1];
-	memory.addr[2] = 0;
-
-	vsp1_rwpf_set_memory(rpf, &memory, false);
+	rpf->mem.addr[0] = mem[0];
+	rpf->mem.addr[1] = mem[1];
+	rpf->mem.addr[2] = 0;
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
@@ -482,14 +473,17 @@ void vsp1_du_atomic_flush(struct device *dev)
 				entity->subdev.name);
 			return;
 		}
+
+		if (entity->type == VSP1_ENTITY_RPF)
+			vsp1_rwpf_set_memory(to_rwpf(&entity->subdev));
 	}
 
 	vsp1_dl_list_commit(pipe->dl);
 	pipe->dl = NULL;
 
+	/* Start or stop the pipeline if needed. */
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
-	/* Start or stop the pipeline if needed. */
 	if (!vsp1->drm->num_inputs && pipe->num_inputs) {
 		vsp1_write(vsp1, VI6_DISP_IRQ_STA, 0);
 		vsp1_write(vsp1, VI6_DISP_IRQ_ENB, VI6_DISP_IRQ_ENB_DSTE);
@@ -569,7 +563,6 @@ int vsp1_drm_init(struct vsp1_device *vsp1)
 	pipe = &vsp1->drm->pipe;
 
 	vsp1_pipeline_init(pipe);
-	pipe->frame_end = vsp1_drm_frame_end;
 
 	/* The DRM pipeline is static, add entities manually. */
 	for (i = 0; i < vsp1->info->rpf_count; ++i) {
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index be67727f6f78..7b2301dbd584 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -27,10 +27,7 @@ void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
 {
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&e->subdev.entity);
 
-	if (pipe->dl)
-		vsp1_dl_list_write(pipe->dl, reg, data);
-	else
-		vsp1_write(e->vsp1, reg, data);
+	vsp1_dl_list_write(pipe->dl, reg, data);
 }
 
 void vsp1_entity_route_setup(struct vsp1_entity *source)
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index a9a754e17e8d..3311db18f40b 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -273,42 +273,13 @@ bool vsp1_pipeline_ready(struct vsp1_pipeline *pipe)
 
 void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
 {
-	enum vsp1_pipeline_state state;
-	unsigned long flags;
-
 	if (pipe == NULL)
 		return;
 
-	/* Signal frame end to the pipeline handler. */
+	vsp1_dlm_irq_frame_end(pipe->output->dlm);
+
 	if (pipe->frame_end)
 		pipe->frame_end(pipe);
-
-	spin_lock_irqsave(&pipe->irqlock, flags);
-
-	state = pipe->state;
-
-	/* When using display lists in continuous frame mode the pipeline is
-	 * automatically restarted by the hardware.
-	 */
-	if (pipe->lif)
-		goto done;
-
-	pipe->state = VSP1_PIPELINE_STOPPED;
-
-	/* If a stop has been requested, mark the pipeline as stopped and
-	 * return.
-	 */
-	if (state == VSP1_PIPELINE_STOPPING) {
-		wake_up(&pipe->wq);
-		goto done;
-	}
-
-	/* Restart the pipeline if ready. */
-	if (vsp1_pipeline_ready(pipe))
-		vsp1_pipeline_run(pipe);
-
-done:
-	spin_unlock_irqrestore(&pipe->irqlock, flags);
 }
 
 /*
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 9857d633f61e..1bbb8d8f9bcb 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -78,9 +78,6 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_PSTRIDE, pstride);
 
-	/* Now that the offsets have been computed program the DMA addresses. */
-	rpf->ops->set_memory(rpf);
-
 	/* Format */
 	infmt = VI6_RPF_INFMT_CIPM
 	      | (fmtinfo->hwfmt << VI6_RPF_INFMT_RDFMT_SHIFT);
@@ -150,11 +147,11 @@ static struct v4l2_subdev_ops rpf_ops = {
 static void rpf_set_memory(struct vsp1_rwpf *rpf)
 {
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
-		       rpf->buf_addr[0] + rpf->offsets[0]);
+		       rpf->mem.addr[0] + rpf->offsets[0]);
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
-		       rpf->buf_addr[1] + rpf->offsets[1]);
+		       rpf->mem.addr[1] + rpf->offsets[1]);
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
-		       rpf->buf_addr[2] + rpf->offsets[1]);
+		       rpf->mem.addr[2] + rpf->offsets[1]);
 }
 
 static const struct vsp1_rwpf_operations rpf_vdev_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 0924079b920c..38893ab06cd9 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -269,29 +269,3 @@ int vsp1_rwpf_init_ctrls(struct vsp1_rwpf *rwpf)
 
 	return rwpf->ctrls.error;
 }
-
-/* -----------------------------------------------------------------------------
- * Buffers
- */
-
-/**
- * vsp1_rwpf_set_memory - Configure DMA addresses for a [RW]PF
- * @rwpf: the [RW]PF instance
- * @mem: DMA memory addresses
- * @apply: whether to apply the configuration to the hardware
- *
- * This function stores the DMA addresses for all planes in the rwpf instance
- * and optionally applies the configuration to hardware registers if the apply
- * argument is set to true.
- */
-void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf, struct vsp1_rwpf_memory *mem,
-			  bool apply)
-{
-	unsigned int i;
-
-	for (i = 0; i < 3; ++i)
-		rwpf->buf_addr[i] = mem->addr[i];
-
-	if (apply)
-		rwpf->ops->set_memory(rwpf);
-}
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 57f15d45f8bb..2bbcc331959b 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -29,15 +29,13 @@ struct vsp1_rwpf;
 struct vsp1_video;
 
 struct vsp1_rwpf_memory {
-	unsigned int num_planes;
 	dma_addr_t addr[3];
-	unsigned int length[3];
 };
 
 /**
  * struct vsp1_rwpf_operations - RPF and WPF operations
  * @set_memory: Setup memory buffer access. This operation applies the settings
- *		stored in the rwpf buf_addr field to the hardware.
+ *		stored in the rwpf mem field to the hardware.
  */
 struct vsp1_rwpf_operations {
 	void (*set_memory)(struct vsp1_rwpf *rwpf);
@@ -65,7 +63,7 @@ struct vsp1_rwpf {
 	unsigned int alpha;
 
 	unsigned int offsets[2];
-	dma_addr_t buf_addr[3];
+	struct vsp1_rwpf_memory mem;
 
 	struct vsp1_dl_manager *dlm;
 };
@@ -99,7 +97,15 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 			    struct v4l2_subdev_pad_config *cfg,
 			    struct v4l2_subdev_selection *sel);
 
-void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf, struct vsp1_rwpf_memory *mem,
-			  bool apply);
+/**
+ * vsp1_rwpf_set_memory - Configure DMA addresses for a [RW]PF
+ * @rwpf: the [RW]PF instance
+ *
+ * This function applies the cached memory buffer address to the hardware.
+ */
+static inline void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf)
+{
+	rwpf->ops->set_memory(rwpf);
+}
 
 #endif /* __VSP1_RWPF_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 96b04fcd33ae..7cb270f57f62 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -29,6 +29,7 @@
 
 #include "vsp1.h"
 #include "vsp1_bru.h"
+#include "vsp1_dl.h"
 #include "vsp1_entity.h"
 #include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
@@ -424,7 +425,7 @@ vsp1_video_complete_buffer(struct vsp1_video *video)
 	done->buf.vb2_buf.timestamp = ktime_get_ns();
 	for (i = 0; i < done->buf.vb2_buf.num_planes; ++i)
 		vb2_set_plane_payload(&done->buf.vb2_buf, i,
-				      done->mem.length[i]);
+				      vb2_plane_size(&done->buf.vb2_buf, i));
 	vb2_buffer_done(&done->buf.vb2_buf, VB2_BUF_STATE_DONE);
 
 	return next;
@@ -443,15 +444,41 @@ static void vsp1_video_frame_end(struct vsp1_pipeline *pipe,
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
-	vsp1_rwpf_set_memory(video->rwpf, &buf->mem, true);
+	video->rwpf->mem = buf->mem;
 	pipe->buffers_ready |= 1 << video->pipe_index;
 
 	spin_unlock_irqrestore(&pipe->irqlock, flags);
 }
 
+static void vsp1_video_pipeline_run(struct vsp1_pipeline *pipe)
+{
+	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+	unsigned int i;
+
+	if (!pipe->dl)
+		pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
+
+	for (i = 0; i < vsp1->info->rpf_count; ++i) {
+		struct vsp1_rwpf *rwpf = pipe->inputs[i];
+
+		if (rwpf)
+			vsp1_rwpf_set_memory(rwpf);
+	}
+
+	if (!pipe->lif)
+		vsp1_rwpf_set_memory(pipe->output);
+
+	vsp1_dl_list_commit(pipe->dl);
+	pipe->dl = NULL;
+
+	vsp1_pipeline_run(pipe);
+}
+
 static void vsp1_video_pipeline_frame_end(struct vsp1_pipeline *pipe)
 {
 	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+	enum vsp1_pipeline_state state;
+	unsigned long flags;
 	unsigned int i;
 
 	/* Complete buffers on all video nodes. */
@@ -462,8 +489,22 @@ static void vsp1_video_pipeline_frame_end(struct vsp1_pipeline *pipe)
 		vsp1_video_frame_end(pipe, pipe->inputs[i]);
 	}
 
-	if (!pipe->lif)
-		vsp1_video_frame_end(pipe, pipe->output);
+	vsp1_video_frame_end(pipe, pipe->output);
+
+	spin_lock_irqsave(&pipe->irqlock, flags);
+
+	state = pipe->state;
+	pipe->state = VSP1_PIPELINE_STOPPED;
+
+	/* If a stop has been requested, mark the pipeline as stopped and
+	 * return. Otherwise restart the pipeline if ready.
+	 */
+	if (state == VSP1_PIPELINE_STOPPING)
+		wake_up(&pipe->wq);
+	else if (vsp1_pipeline_ready(pipe))
+		vsp1_video_pipeline_run(pipe);
+
+	spin_unlock_irqrestore(&pipe->irqlock, flags);
 }
 
 /* -----------------------------------------------------------------------------
@@ -512,20 +553,15 @@ static int vsp1_video_buffer_prepare(struct vb2_buffer *vb)
 	if (vb->num_planes < format->num_planes)
 		return -EINVAL;
 
-	buf->mem.num_planes = vb->num_planes;
-
 	for (i = 0; i < vb->num_planes; ++i) {
 		buf->mem.addr[i] = vb2_dma_contig_plane_dma_addr(vb, i);
-		buf->mem.length[i] = vb2_plane_size(vb, i);
 
-		if (buf->mem.length[i] < format->plane_fmt[i].sizeimage)
+		if (vb2_plane_size(vb, i) < format->plane_fmt[i].sizeimage)
 			return -EINVAL;
 	}
 
-	for ( ; i < 3; ++i) {
+	for ( ; i < 3; ++i)
 		buf->mem.addr[i] = 0;
-		buf->mem.length[i] = 0;
-	}
 
 	return 0;
 }
@@ -549,54 +585,74 @@ static void vsp1_video_buffer_queue(struct vb2_buffer *vb)
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 
-	vsp1_rwpf_set_memory(video->rwpf, &buf->mem, true);
+	video->rwpf->mem = buf->mem;
 	pipe->buffers_ready |= 1 << video->pipe_index;
 
 	if (vb2_is_streaming(&video->queue) &&
 	    vsp1_pipeline_ready(pipe))
-		vsp1_pipeline_run(pipe);
+		vsp1_video_pipeline_run(pipe);
 
 	spin_unlock_irqrestore(&pipe->irqlock, flags);
 }
 
+static int vsp1_video_setup_pipeline(struct vsp1_pipeline *pipe)
+{
+	struct vsp1_entity *entity;
+	int ret;
+
+	/* Prepare the display list. */
+	pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
+	if (!pipe->dl)
+		return -ENOMEM;
+
+	if (pipe->uds) {
+		struct vsp1_uds *uds = to_uds(&pipe->uds->subdev);
+
+		/* If a BRU is present in the pipeline before the UDS, the alpha
+		 * component doesn't need to be scaled as the BRU output alpha
+		 * value is fixed to 255. Otherwise we need to scale the alpha
+		 * component only when available at the input RPF.
+		 */
+		if (pipe->uds_input->type == VSP1_ENTITY_BRU) {
+			uds->scale_alpha = false;
+		} else {
+			struct vsp1_rwpf *rpf =
+				to_rwpf(&pipe->uds_input->subdev);
+
+			uds->scale_alpha = rpf->fmtinfo->alpha;
+		}
+	}
+
+	list_for_each_entry(entity, &pipe->entities, list_pipe) {
+		vsp1_entity_route_setup(entity);
+
+		ret = v4l2_subdev_call(&entity->subdev, video, s_stream, 1);
+		if (ret < 0)
+			goto error;
+	}
+
+	return 0;
+
+error:
+	vsp1_dl_list_put(pipe->dl);
+	pipe->dl = NULL;
+
+	return ret;
+}
+
 static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct vsp1_video *video = vb2_get_drv_priv(vq);
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
-	struct vsp1_entity *entity;
 	unsigned long flags;
 	int ret;
 
 	mutex_lock(&pipe->lock);
 	if (pipe->stream_count == pipe->num_inputs) {
-		if (pipe->uds) {
-			struct vsp1_uds *uds = to_uds(&pipe->uds->subdev);
-
-			/* If a BRU is present in the pipeline before the UDS,
-			 * the alpha component doesn't need to be scaled as the
-			 * BRU output alpha value is fixed to 255. Otherwise we
-			 * need to scale the alpha component only when available
-			 * at the input RPF.
-			 */
-			if (pipe->uds_input->type == VSP1_ENTITY_BRU) {
-				uds->scale_alpha = false;
-			} else {
-				struct vsp1_rwpf *rpf =
-					to_rwpf(&pipe->uds_input->subdev);
-
-				uds->scale_alpha = rpf->fmtinfo->alpha;
-			}
-		}
-
-		list_for_each_entry(entity, &pipe->entities, list_pipe) {
-			vsp1_entity_route_setup(entity);
-
-			ret = v4l2_subdev_call(&entity->subdev, video,
-					       s_stream, 1);
-			if (ret < 0) {
-				mutex_unlock(&pipe->lock);
-				return ret;
-			}
+		ret = vsp1_video_setup_pipeline(pipe);
+		if (ret < 0) {
+			mutex_unlock(&pipe->lock);
+			return ret;
 		}
 	}
 
@@ -605,7 +661,7 @@ static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
 
 	spin_lock_irqsave(&pipe->irqlock, flags);
 	if (vsp1_pipeline_ready(pipe))
-		vsp1_pipeline_run(pipe);
+		vsp1_video_pipeline_run(pipe);
 	spin_unlock_irqrestore(&pipe->irqlock, flags);
 
 	return 0;
@@ -625,6 +681,9 @@ static void vsp1_video_stop_streaming(struct vb2_queue *vq)
 		ret = vsp1_pipeline_stop(pipe);
 		if (ret == -ETIMEDOUT)
 			dev_err(video->vsp1->dev, "pipeline stop timeout\n");
+
+		vsp1_dl_list_put(pipe->dl);
+		pipe->dl = NULL;
 	}
 	mutex_unlock(&pipe->lock);
 
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index d82502681bc3..08c8fce23098 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -157,9 +157,9 @@ static struct v4l2_subdev_ops wpf_ops = {
 
 static void wpf_set_memory(struct vsp1_rwpf *wpf)
 {
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, wpf->buf_addr[0]);
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, wpf->buf_addr[1]);
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, wpf->buf_addr[2]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, wpf->mem.addr[0]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, wpf->mem.addr[1]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
 }
 
 static const struct vsp1_rwpf_operations wpf_vdev_ops = {
@@ -200,13 +200,11 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the display list manager if the WPF is used for display */
-	if ((vsp1->info->features & VSP1_HAS_LIF) && index == 0) {
-		wpf->dlm = vsp1_dlm_create(vsp1, index, 4);
-		if (!wpf->dlm) {
-			ret = -ENOMEM;
-			goto error;
-		}
+	/* Initialize the display list manager. */
+	wpf->dlm = vsp1_dlm_create(vsp1, index, 4);
+	if (!wpf->dlm) {
+		ret = -ENOMEM;
+		goto error;
 	}
 
 	/* Initialize the V4L2 subdev. */
-- 
2.7.3

[PATCH v2 25/54] v4l: vsp1: Move subdev initialization code to vsp1_entity_init()

[Laurent Pinchart]

Don't duplicate the code in every module driver, centralize it in a
single place.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 21 +++------------------
 drivers/media/platform/vsp1/vsp1_entity.c | 31 ++++++++++++++++++++++++++-----
 drivers/media/platform/vsp1/vsp1_entity.h |  5 ++---
 drivers/media/platform/vsp1/vsp1_hsit.c   | 19 ++-----------------
 drivers/media/platform/vsp1/vsp1_lif.c    | 19 ++-----------------
 drivers/media/platform/vsp1/vsp1_lut.c    | 19 ++-----------------
 drivers/media/platform/vsp1/vsp1_rpf.c    | 21 ++++-----------------
 drivers/media/platform/vsp1/vsp1_sru.c    | 19 ++-----------------
 drivers/media/platform/vsp1/vsp1_uds.c    | 21 ++++-----------------
 drivers/media/platform/vsp1/vsp1_wpf.c    | 21 ++++-----------------
 10 files changed, 51 insertions(+), 145 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index fe6ebfa83311..77da872b08c5 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -405,7 +405,6 @@ static struct v4l2_subdev_ops bru_ops = {
 
 struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_bru *bru;
 	int ret;
 
@@ -415,26 +414,12 @@ struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1)
 
 	bru->entity.type = VSP1_ENTITY_BRU;
 
-	ret = vsp1_entity_init(vsp1, &bru->entity,
-			       vsp1->info->num_bru_inputs + 1);
+	ret = vsp1_entity_init(vsp1, &bru->entity, "bru",
+			       vsp1->info->num_bru_inputs + 1, &bru_ops,
+			       MEDIA_ENT_F_PROC_VIDEO_COMPOSER);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &bru->entity.subdev;
-	v4l2_subdev_init(subdev, &bru_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_COMPOSER;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s bru",
-		 dev_name(vsp1->dev));
-	v4l2_set_subdevdata(subdev, bru);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	/* Initialize the control handler. */
 	v4l2_ctrl_handler_init(&bru->ctrls, 1);
 	v4l2_ctrl_new_std(&bru->ctrls, &bru_ctrl_ops, V4L2_CID_BG_COLOR,
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 7b2301dbd584..dcb331fb5549 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -70,8 +70,8 @@ vsp1_entity_get_pad_format(struct vsp1_entity *entity,
  * formats are initialized on the file handle. Otherwise active formats are
  * initialized on the device.
  */
-void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
-			    struct v4l2_subdev_pad_config *cfg)
+static void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
+				     struct v4l2_subdev_pad_config *cfg)
 {
 	struct v4l2_subdev_format format;
 	unsigned int pad;
@@ -159,9 +159,12 @@ static const struct vsp1_route vsp1_routes[] = {
 };
 
 int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
-		     unsigned int num_pads)
+		     const char *name, unsigned int num_pads,
+		     const struct v4l2_subdev_ops *ops, u32 function)
 {
+	struct v4l2_subdev *subdev;
 	unsigned int i;
+	int ret;
 
 	for (i = 0; i < ARRAY_SIZE(vsp1_routes); ++i) {
 		if (vsp1_routes[i].type == entity->type &&
@@ -196,8 +199,26 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 	entity->pads[num_pads - 1].flags = MEDIA_PAD_FL_SOURCE;
 
 	/* Initialize the media entity. */
-	return media_entity_pads_init(&entity->subdev.entity, num_pads,
-				 entity->pads);
+	ret = media_entity_pads_init(&entity->subdev.entity, num_pads,
+				     entity->pads);
+	if (ret < 0)
+		return ret;
+
+	/* Initialize the V4L2 subdev. */
+	subdev = &entity->subdev;
+	v4l2_subdev_init(subdev, ops);
+
+	subdev->entity.function = function;
+	subdev->entity.ops = &vsp1->media_ops;
+	subdev->internal_ops = &vsp1_subdev_internal_ops;
+	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
+
+	snprintf(subdev->name, sizeof(subdev->name), "%s %s",
+		 dev_name(vsp1->dev), name);
+
+	vsp1_entity_init_formats(subdev, NULL);
+
+	return 0;
 }
 
 void vsp1_entity_destroy(struct vsp1_entity *entity)
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 203872164f8e..056391105ee5 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -81,7 +81,8 @@ static inline struct vsp1_entity *to_vsp1_entity(struct v4l2_subdev *subdev)
 }
 
 int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
-		     unsigned int num_pads);
+		     const char *name, unsigned int num_pads,
+		     const struct v4l2_subdev_ops *ops, u32 function);
 void vsp1_entity_destroy(struct vsp1_entity *entity);
 
 extern const struct v4l2_subdev_internal_ops vsp1_subdev_internal_ops;
@@ -94,8 +95,6 @@ struct v4l2_mbus_framefmt *
 vsp1_entity_get_pad_format(struct vsp1_entity *entity,
 			   struct v4l2_subdev_pad_config *cfg,
 			   unsigned int pad, u32 which);
-void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
-			      struct v4l2_subdev_pad_config *cfg);
 
 void vsp1_entity_route_setup(struct vsp1_entity *source);
 
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index e971dfa9714d..457716cbb7ab 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -180,7 +180,6 @@ static struct v4l2_subdev_ops hsit_ops = {
 
 struct vsp1_hsit *vsp1_hsit_create(struct vsp1_device *vsp1, bool inverse)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_hsit *hsit;
 	int ret;
 
@@ -195,24 +194,10 @@ struct vsp1_hsit *vsp1_hsit_create(struct vsp1_device *vsp1, bool inverse)
 	else
 		hsit->entity.type = VSP1_ENTITY_HST;
 
-	ret = vsp1_entity_init(vsp1, &hsit->entity, 2);
+	ret = vsp1_entity_init(vsp1, &hsit->entity, inverse ? "hsi" : "hst", 2,
+			       &hsit_ops, MEDIA_ENT_F_PROC_VIDEO_CONVERTER);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &hsit->entity.subdev;
-	v4l2_subdev_init(subdev, &hsit_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_CONVERTER;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s %s",
-		 dev_name(vsp1->dev), inverse ? "hsi" : "hst");
-	v4l2_set_subdevdata(subdev, hsit);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	return hsit;
 }
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 53725769442d..0cb58c3576ed 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -207,7 +207,6 @@ static struct v4l2_subdev_ops lif_ops = {
 
 struct vsp1_lif *vsp1_lif_create(struct vsp1_device *vsp1)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_lif *lif;
 	int ret;
 
@@ -217,24 +216,10 @@ struct vsp1_lif *vsp1_lif_create(struct vsp1_device *vsp1)
 
 	lif->entity.type = VSP1_ENTITY_LIF;
 
-	ret = vsp1_entity_init(vsp1, &lif->entity, 2);
+	ret = vsp1_entity_init(vsp1, &lif->entity, "lif", 2, &lif_ops,
+			       MEDIA_ENT_F_PROC_VIDEO_GENERIC);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &lif->entity.subdev;
-	v4l2_subdev_init(subdev, &lif_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_GENERIC;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s lif",
-		 dev_name(vsp1->dev));
-	v4l2_set_subdevdata(subdev, lif);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	return lif;
 }
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 1c0eefeee2f3..491275fe6953 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -221,7 +221,6 @@ static struct v4l2_subdev_ops lut_ops = {
 
 struct vsp1_lut *vsp1_lut_create(struct vsp1_device *vsp1)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_lut *lut;
 	int ret;
 
@@ -231,24 +230,10 @@ struct vsp1_lut *vsp1_lut_create(struct vsp1_device *vsp1)
 
 	lut->entity.type = VSP1_ENTITY_LUT;
 
-	ret = vsp1_entity_init(vsp1, &lut->entity, 2);
+	ret = vsp1_entity_init(vsp1, &lut->entity, "lut", 2, &lut_ops,
+			       MEDIA_ENT_F_PROC_VIDEO_GENERIC);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &lut->entity.subdev;
-	v4l2_subdev_init(subdev, &lut_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_GENERIC;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s lut",
-		 dev_name(vsp1->dev));
-	v4l2_set_subdevdata(subdev, lut);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	return lut;
 }
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 1bbb8d8f9bcb..d9cc9b8fc0b6 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -164,8 +164,8 @@ static const struct vsp1_rwpf_operations rpf_vdev_ops = {
 
 struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_rwpf *rpf;
+	char name[6];
 	int ret;
 
 	rpf = devm_kzalloc(vsp1->dev, sizeof(*rpf), GFP_KERNEL);
@@ -180,25 +180,12 @@ struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index)
 	rpf->entity.type = VSP1_ENTITY_RPF;
 	rpf->entity.index = index;
 
-	ret = vsp1_entity_init(vsp1, &rpf->entity, 2);
+	sprintf(name, "rpf.%u", index);
+	ret = vsp1_entity_init(vsp1, &rpf->entity, name, 2, &rpf_ops,
+			       MEDIA_ENT_F_PROC_VIDEO_CONVERTER);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &rpf->entity.subdev;
-	v4l2_subdev_init(subdev, &rpf_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_CONVERTER;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s rpf.%u",
-		 dev_name(vsp1->dev), index);
-	v4l2_set_subdevdata(subdev, rpf);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	/* Initialize the control handler. */
 	ret = vsp1_rwpf_init_ctrls(rpf);
 	if (ret < 0) {
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index d81a5b6300de..ce6a97cb0ec1 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -324,7 +324,6 @@ static struct v4l2_subdev_ops sru_ops = {
 
 struct vsp1_sru *vsp1_sru_create(struct vsp1_device *vsp1)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_sru *sru;
 	int ret;
 
@@ -334,25 +333,11 @@ struct vsp1_sru *vsp1_sru_create(struct vsp1_device *vsp1)
 
 	sru->entity.type = VSP1_ENTITY_SRU;
 
-	ret = vsp1_entity_init(vsp1, &sru->entity, 2);
+	ret = vsp1_entity_init(vsp1, &sru->entity, "sru", 2, &sru_ops,
+			       MEDIA_ENT_F_PROC_VIDEO_SCALER);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &sru->entity.subdev;
-	v4l2_subdev_init(subdev, &sru_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s sru",
-		 dev_name(vsp1->dev));
-	v4l2_set_subdevdata(subdev, sru);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	/* Initialize the control handler. */
 	v4l2_ctrl_handler_init(&sru->ctrls, 1);
 	v4l2_ctrl_new_custom(&sru->ctrls, &sru_intensity_control, NULL);
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 286d0daea810..4266c079ada0 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -322,8 +322,8 @@ static struct v4l2_subdev_ops uds_ops = {
 
 struct vsp1_uds *vsp1_uds_create(struct vsp1_device *vsp1, unsigned int index)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_uds *uds;
+	char name[6];
 	int ret;
 
 	uds = devm_kzalloc(vsp1->dev, sizeof(*uds), GFP_KERNEL);
@@ -333,24 +333,11 @@ struct vsp1_uds *vsp1_uds_create(struct vsp1_device *vsp1, unsigned int index)
 	uds->entity.type = VSP1_ENTITY_UDS;
 	uds->entity.index = index;
 
-	ret = vsp1_entity_init(vsp1, &uds->entity, 2);
+	sprintf(name, "uds.%u", index);
+	ret = vsp1_entity_init(vsp1, &uds->entity, name, 2, &uds_ops,
+			       MEDIA_ENT_F_PROC_VIDEO_SCALER);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &uds->entity.subdev;
-	v4l2_subdev_init(subdev, &uds_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s uds.%u",
-		 dev_name(vsp1->dev), index);
-	v4l2_set_subdevdata(subdev, uds);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	return uds;
 }
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 08c8fce23098..f4861a2d2002 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -179,8 +179,8 @@ static void vsp1_wpf_destroy(struct vsp1_entity *entity)
 
 struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 {
-	struct v4l2_subdev *subdev;
 	struct vsp1_rwpf *wpf;
+	char name[6];
 	int ret;
 
 	wpf = devm_kzalloc(vsp1->dev, sizeof(*wpf), GFP_KERNEL);
@@ -196,7 +196,9 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	wpf->entity.type = VSP1_ENTITY_WPF;
 	wpf->entity.index = index;
 
-	ret = vsp1_entity_init(vsp1, &wpf->entity, 2);
+	sprintf(name, "wpf.%u", index);
+	ret = vsp1_entity_init(vsp1, &wpf->entity, name, 2, &wpf_ops,
+			       MEDIA_ENT_F_PROC_VIDEO_CONVERTER);
 	if (ret < 0)
 		return ERR_PTR(ret);
 
@@ -207,21 +209,6 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 		goto error;
 	}
 
-	/* Initialize the V4L2 subdev. */
-	subdev = &wpf->entity.subdev;
-	v4l2_subdev_init(subdev, &wpf_ops);
-
-	subdev->entity.function = MEDIA_ENT_F_PROC_VIDEO_CONVERTER;
-	subdev->entity.ops = &vsp1->media_ops;
-
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
-	snprintf(subdev->name, sizeof(subdev->name), "%s wpf.%u",
-		 dev_name(vsp1->dev), index);
-	v4l2_set_subdevdata(subdev, wpf);
-	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
-
-	vsp1_entity_init_formats(subdev, NULL);
-
 	/* Initialize the control handler. */
 	ret = vsp1_rwpf_init_ctrls(wpf);
 	if (ret < 0) {
-- 
2.7.3

[PATCH v2 26/54] v4l: vsp1: Consolidate entity ops in a struct vsp1_entity_operations

[Laurent Pinchart]

Entities have two operations, a destroy operation stored directly in
vsp1_entity and a set_memory operation stored in a vsp1_rwpf_operations
structure. Move the two to a more generic vsp1_entity_operations
structure that will serve to implement additional operations.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_entity.c |  4 ++--
 drivers/media/platform/vsp1/vsp1_entity.h | 14 +++++++++++++-
 drivers/media/platform/vsp1/vsp1_rpf.c    | 11 ++++++-----
 drivers/media/platform/vsp1/vsp1_rwpf.h   | 18 ++++++------------
 drivers/media/platform/vsp1/vsp1_wpf.c    | 27 ++++++++++++++-------------
 5 files changed, 41 insertions(+), 33 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index dcb331fb5549..f09a54b396ec 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -223,8 +223,8 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 
 void vsp1_entity_destroy(struct vsp1_entity *entity)
 {
-	if (entity->destroy)
-		entity->destroy(entity);
+	if (entity->ops && entity->ops->destroy)
+		entity->ops->destroy(entity);
 	if (entity->subdev.ctrl_handler)
 		v4l2_ctrl_handler_free(entity->subdev.ctrl_handler);
 	media_entity_cleanup(&entity->subdev.entity);
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 056391105ee5..f46ba20c30b1 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -53,10 +53,22 @@ struct vsp1_route {
 	unsigned int inputs[VSP1_ENTITY_MAX_INPUTS];
 };
 
+/**
+ * struct vsp1_entity_operations - Entity operations
+ * @destroy:	Destroy the entity.
+ * @set_memory:	Setup memory buffer access. This operation applies the settings
+ *		stored in the rwpf mem field to the hardware. Valid for RPF and
+ *		WPF only.
+ */
+struct vsp1_entity_operations {
+	void (*destroy)(struct vsp1_entity *);
+	void (*set_memory)(struct vsp1_entity *);
+};
+
 struct vsp1_entity {
 	struct vsp1_device *vsp1;
 
-	void (*destroy)(struct vsp1_entity *);
+	const struct vsp1_entity_operations *ops;
 
 	enum vsp1_entity_type type;
 	unsigned int index;
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index d9cc9b8fc0b6..5c84a92c975c 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -141,11 +141,13 @@ static struct v4l2_subdev_ops rpf_ops = {
 };
 
 /* -----------------------------------------------------------------------------
- * Video Device Operations
+ * VSP1 Entity Operations
  */
 
-static void rpf_set_memory(struct vsp1_rwpf *rpf)
+static void rpf_set_memory(struct vsp1_entity *entity)
 {
+	struct vsp1_rwpf *rpf = entity_to_rwpf(entity);
+
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
 		       rpf->mem.addr[0] + rpf->offsets[0]);
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
@@ -154,7 +156,7 @@ static void rpf_set_memory(struct vsp1_rwpf *rpf)
 		       rpf->mem.addr[2] + rpf->offsets[1]);
 }
 
-static const struct vsp1_rwpf_operations rpf_vdev_ops = {
+static const struct vsp1_entity_operations rpf_entity_ops = {
 	.set_memory = rpf_set_memory,
 };
 
@@ -172,11 +174,10 @@ struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index)
 	if (rpf == NULL)
 		return ERR_PTR(-ENOMEM);
 
-	rpf->ops = &rpf_vdev_ops;
-
 	rpf->max_width = RPF_MAX_WIDTH;
 	rpf->max_height = RPF_MAX_HEIGHT;
 
+	rpf->entity.ops = &rpf_entity_ops;
 	rpf->entity.type = VSP1_ENTITY_RPF;
 	rpf->entity.index = index;
 
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 2bbcc331959b..e8ca9b6ee689 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -32,23 +32,12 @@ struct vsp1_rwpf_memory {
 	dma_addr_t addr[3];
 };
 
-/**
- * struct vsp1_rwpf_operations - RPF and WPF operations
- * @set_memory: Setup memory buffer access. This operation applies the settings
- *		stored in the rwpf mem field to the hardware.
- */
-struct vsp1_rwpf_operations {
-	void (*set_memory)(struct vsp1_rwpf *rwpf);
-};
-
 struct vsp1_rwpf {
 	struct vsp1_entity entity;
 	struct v4l2_ctrl_handler ctrls;
 
 	struct vsp1_video *video;
 
-	const struct vsp1_rwpf_operations *ops;
-
 	unsigned int max_width;
 	unsigned int max_height;
 
@@ -73,6 +62,11 @@ static inline struct vsp1_rwpf *to_rwpf(struct v4l2_subdev *subdev)
 	return container_of(subdev, struct vsp1_rwpf, entity.subdev);
 }
 
+static inline struct vsp1_rwpf *entity_to_rwpf(struct vsp1_entity *entity)
+{
+	return container_of(entity, struct vsp1_rwpf, entity);
+}
+
 struct vsp1_rwpf *vsp1_rpf_create(struct vsp1_device *vsp1, unsigned int index);
 struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index);
 
@@ -105,7 +99,7 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
  */
 static inline void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf)
 {
-	rwpf->ops->set_memory(rwpf);
+	rwpf->entity.ops->set_memory(&rwpf->entity);
 }
 
 #endif /* __VSP1_RWPF_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index f4861a2d2002..8cc19ef49f45 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -152,17 +152,27 @@ static struct v4l2_subdev_ops wpf_ops = {
 };
 
 /* -----------------------------------------------------------------------------
- * Video Device Operations
+ * VSP1 Entity Operations
  */
 
-static void wpf_set_memory(struct vsp1_rwpf *wpf)
+static void vsp1_wpf_destroy(struct vsp1_entity *entity)
 {
+	struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+
+	vsp1_dlm_destroy(wpf->dlm);
+}
+
+static void wpf_set_memory(struct vsp1_entity *entity)
+{
+	struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+
 	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, wpf->mem.addr[0]);
 	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, wpf->mem.addr[1]);
 	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
 }
 
-static const struct vsp1_rwpf_operations wpf_vdev_ops = {
+static const struct vsp1_entity_operations wpf_entity_ops = {
+	.destroy = vsp1_wpf_destroy,
 	.set_memory = wpf_set_memory,
 };
 
@@ -170,13 +180,6 @@ static const struct vsp1_rwpf_operations wpf_vdev_ops = {
  * Initialization and Cleanup
  */
 
-static void vsp1_wpf_destroy(struct vsp1_entity *entity)
-{
-	struct vsp1_rwpf *wpf = container_of(entity, struct vsp1_rwpf, entity);
-
-	vsp1_dlm_destroy(wpf->dlm);
-}
-
 struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 {
 	struct vsp1_rwpf *wpf;
@@ -187,12 +190,10 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	if (wpf == NULL)
 		return ERR_PTR(-ENOMEM);
 
-	wpf->ops = &wpf_vdev_ops;
-
 	wpf->max_width = WPF_MAX_WIDTH;
 	wpf->max_height = WPF_MAX_HEIGHT;
 
-	wpf->entity.destroy = vsp1_wpf_destroy;
+	wpf->entity.ops = &wpf_entity_ops;
 	wpf->entity.type = VSP1_ENTITY_WPF;
 	wpf->entity.index = index;
 
-- 
2.7.3

[PATCH v2 27/54] v4l: vsp1: Fix BRU try compose rectangle storage

[Laurent Pinchart]

Fix a typo that stored the try compose rectangle in the crop rectangle.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 77da872b08c5..94679fec3864 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -228,7 +228,8 @@ static struct v4l2_rect *bru_get_compose(struct vsp1_bru *bru,
 {
 	switch (which) {
 	case V4L2_SUBDEV_FORMAT_TRY:
-		return v4l2_subdev_get_try_crop(&bru->entity.subdev, cfg, pad);
+		return v4l2_subdev_get_try_compose(&bru->entity.subdev, cfg,
+						   pad);
 	case V4L2_SUBDEV_FORMAT_ACTIVE:
 		return &bru->inputs[pad].compose;
 	default:
-- 
2.7.3

[PATCH v2 28/54] v4l: vsp1: Add race condition FIXME comment

[Laurent Pinchart]

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_video.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 7cb270f57f62..102977ae1daa 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -813,6 +813,9 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	 *
 	 * Use the VSP1 pipeline object embedded in the first video object that
 	 * starts streaming.
+	 *
+	 * FIXME: This is racy, the ioctl is only protected by the video node
+	 * lock.
 	 */
 	pipe = video->video.entity.pipe
 	     ? to_vsp1_pipeline(&video->video.entity) : &video->pipe;
-- 
2.7.3

[PATCH v2 29/54] v4l: vsp1: Implement and use the subdev pad::init_cfg configuration

[Laurent Pinchart]

Turn the custom formats initialization function into a standard
pad::init_cfg handler and use it in subdevs instead of initializing
formats in the subdev open handler.

This makes the subdev open handler empty, so remove it.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    |  1 +
 drivers/media/platform/vsp1/vsp1_entity.c | 24 ++++++------------------
 drivers/media/platform/vsp1/vsp1_entity.h |  2 ++
 drivers/media/platform/vsp1/vsp1_hsit.c   |  1 +
 drivers/media/platform/vsp1/vsp1_lif.c    |  1 +
 drivers/media/platform/vsp1/vsp1_lut.c    |  1 +
 drivers/media/platform/vsp1/vsp1_rpf.c    |  1 +
 drivers/media/platform/vsp1/vsp1_sru.c    |  1 +
 drivers/media/platform/vsp1/vsp1_uds.c    |  1 +
 drivers/media/platform/vsp1/vsp1_wpf.c    |  1 +
 10 files changed, 16 insertions(+), 18 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 94679fec3864..7dcd07027bcc 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -387,6 +387,7 @@ static struct v4l2_subdev_video_ops bru_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops bru_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = bru_enum_mbus_code,
 	.enum_frame_size = bru_enum_frame_size,
 	.get_fmt = bru_get_format,
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index f09a54b396ec..3bd55d5ca739 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -62,16 +62,15 @@ vsp1_entity_get_pad_format(struct vsp1_entity *entity,
 }
 
 /*
- * vsp1_entity_init_formats - Initialize formats on all pads
+ * vsp1_entity_init_cfg - Initialize formats on all pads
  * @subdev: V4L2 subdevice
  * @cfg: V4L2 subdev pad configuration
  *
- * Initialize all pad formats with default values. If cfg is not NULL, try
- * formats are initialized on the file handle. Otherwise active formats are
- * initialized on the device.
+ * Initialize all pad formats with default values in the given pad config. This
+ * function can be used as a handler for the subdev pad::init_cfg operation.
  */
-static void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
-				     struct v4l2_subdev_pad_config *cfg)
+int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
+			 struct v4l2_subdev_pad_config *cfg)
 {
 	struct v4l2_subdev_format format;
 	unsigned int pad;
@@ -85,20 +84,10 @@ static void vsp1_entity_init_formats(struct v4l2_subdev *subdev,
 
 		v4l2_subdev_call(subdev, pad, set_fmt, cfg, &format);
 	}
-}
-
-static int vsp1_entity_open(struct v4l2_subdev *subdev,
-			    struct v4l2_subdev_fh *fh)
-{
-	vsp1_entity_init_formats(subdev, fh->pad);
 
 	return 0;
 }
 
-const struct v4l2_subdev_internal_ops vsp1_subdev_internal_ops = {
-	.open = vsp1_entity_open,
-};
-
 /* -----------------------------------------------------------------------------
  * Media Operations
  */
@@ -210,13 +199,12 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 
 	subdev->entity.function = function;
 	subdev->entity.ops = &vsp1->media_ops;
-	subdev->internal_ops = &vsp1_subdev_internal_ops;
 	subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE;
 
 	snprintf(subdev->name, sizeof(subdev->name), "%s %s",
 		 dev_name(vsp1->dev), name);
 
-	vsp1_entity_init_formats(subdev, NULL);
+	vsp1_entity_init_cfg(subdev, NULL);
 
 	return 0;
 }
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index f46ba20c30b1..b608eef9700c 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -107,6 +107,8 @@ struct v4l2_mbus_framefmt *
 vsp1_entity_get_pad_format(struct vsp1_entity *entity,
 			   struct v4l2_subdev_pad_config *cfg,
 			   unsigned int pad, u32 which);
+int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
+			 struct v4l2_subdev_pad_config *cfg);
 
 void vsp1_entity_route_setup(struct vsp1_entity *source);
 
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index 457716cbb7ab..f20293d6587b 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -163,6 +163,7 @@ static struct v4l2_subdev_video_ops hsit_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops hsit_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = hsit_enum_mbus_code,
 	.enum_frame_size = hsit_enum_frame_size,
 	.get_fmt = hsit_get_format,
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 0cb58c3576ed..f7e51ddb8321 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -190,6 +190,7 @@ static struct v4l2_subdev_video_ops lif_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops lif_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = lif_enum_mbus_code,
 	.enum_frame_size = lif_enum_frame_size,
 	.get_fmt = lif_get_format,
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 491275fe6953..9448c4c376aa 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -203,6 +203,7 @@ static struct v4l2_subdev_video_ops lut_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops lut_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = lut_enum_mbus_code,
 	.enum_frame_size = lut_enum_frame_size,
 	.get_fmt = lut_get_format,
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 5c84a92c975c..51542bccd450 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -127,6 +127,7 @@ static struct v4l2_subdev_video_ops rpf_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops rpf_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
 	.enum_frame_size = vsp1_rwpf_enum_frame_size,
 	.get_fmt = vsp1_rwpf_get_format,
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index ce6a97cb0ec1..6eae5a493bc3 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -307,6 +307,7 @@ static struct v4l2_subdev_video_ops sru_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops sru_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = sru_enum_mbus_code,
 	.enum_frame_size = sru_enum_frame_size,
 	.get_fmt = sru_get_format,
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 4266c079ada0..6b6d92c5c629 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -305,6 +305,7 @@ static struct v4l2_subdev_video_ops uds_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops uds_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = uds_enum_mbus_code,
 	.enum_frame_size = uds_enum_frame_size,
 	.get_fmt = uds_get_format,
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 8cc19ef49f45..7f6a7cb00706 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -138,6 +138,7 @@ static struct v4l2_subdev_video_ops wpf_video_ops = {
 };
 
 static struct v4l2_subdev_pad_ops wpf_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
 	.enum_frame_size = vsp1_rwpf_enum_frame_size,
 	.get_fmt = vsp1_rwpf_get_format,
-- 
2.7.3

[PATCH v2 30/54] v4l: vsp1: Store active formats in a pad config structure

[Laurent Pinchart]

Add a pad config structure field to the vsp1_entity structure and use it
to store all active pad formats. This generalizes the code to operate on
pad config structures.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 59 ++++++++++++++++++++----------
 drivers/media/platform/vsp1/vsp1_entity.c | 60 ++++++++++++++++++++++--------
 drivers/media/platform/vsp1/vsp1_entity.h |  8 +++-
 drivers/media/platform/vsp1/vsp1_hsit.c   | 29 +++++++++++----
 drivers/media/platform/vsp1/vsp1_lif.c    | 42 +++++++++++++++------
 drivers/media/platform/vsp1/vsp1_lut.c    | 42 +++++++++++++++------
 drivers/media/platform/vsp1/vsp1_rpf.c    | 12 +++++-
 drivers/media/platform/vsp1/vsp1_rwpf.c   | 51 +++++++++++++++++++-------
 drivers/media/platform/vsp1/vsp1_sru.c    | 61 +++++++++++++++++++++----------
 drivers/media/platform/vsp1/vsp1_uds.c    | 59 ++++++++++++++++++++----------
 drivers/media/platform/vsp1/vsp1_wpf.c    | 12 +++++-
 11 files changed, 311 insertions(+), 124 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 7dcd07027bcc..1691d13131a9 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -70,7 +70,8 @@ static int bru_s_stream(struct v4l2_subdev *subdev, int enable)
 	if (!enable)
 		return 0;
 
-	format = &bru->entity.formats[bru->entity.source_pad];
+	format = vsp1_entity_get_pad_format(&bru->entity, bru->entity.config,
+					    bru->entity.source_pad);
 
 	/* The hardware is extremely flexible but we have no userspace API to
 	 * expose all the parameters, nor is it clear whether we would have use
@@ -183,7 +184,6 @@ static int bru_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
 	struct vsp1_bru *bru = to_bru(subdev);
-	struct v4l2_mbus_framefmt *format;
 
 	if (code->pad == BRU_PAD_SINK(0)) {
 		if (code->index >= ARRAY_SIZE(codes))
@@ -191,12 +191,19 @@ static int bru_enum_mbus_code(struct v4l2_subdev *subdev,
 
 		code->code = codes[code->index];
 	} else {
+		struct v4l2_subdev_pad_config *config;
+		struct v4l2_mbus_framefmt *format;
+
 		if (code->index)
 			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&bru->entity, cfg,
-						    BRU_PAD_SINK(0),
+		config = vsp1_entity_get_pad_config(&bru->entity, cfg,
 						    code->which);
+		if (!config)
+			return -EINVAL;
+
+		format = vsp1_entity_get_pad_format(&bru->entity, config,
+						    BRU_PAD_SINK(0));
 		code->code = format->code;
 	}
 
@@ -242,17 +249,21 @@ static int bru_get_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_bru *bru = to_bru(subdev);
+	struct v4l2_subdev_pad_config *config;
+
+	config = vsp1_entity_get_pad_config(&bru->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
 
-	fmt->format = *vsp1_entity_get_pad_format(&bru->entity, cfg, fmt->pad,
-						  fmt->which);
+	fmt->format = *vsp1_entity_get_pad_format(&bru->entity, config,
+						  fmt->pad);
 
 	return 0;
 }
 
 static void bru_try_format(struct vsp1_bru *bru,
-			   struct v4l2_subdev_pad_config *cfg,
-			   unsigned int pad, struct v4l2_mbus_framefmt *fmt,
-			   enum v4l2_subdev_format_whence which)
+			   struct v4l2_subdev_pad_config *config,
+			   unsigned int pad, struct v4l2_mbus_framefmt *fmt)
 {
 	struct v4l2_mbus_framefmt *format;
 
@@ -266,8 +277,8 @@ static void bru_try_format(struct vsp1_bru *bru,
 
 	default:
 		/* The BRU can't perform format conversion. */
-		format = vsp1_entity_get_pad_format(&bru->entity, cfg,
-						    BRU_PAD_SINK(0), which);
+		format = vsp1_entity_get_pad_format(&bru->entity, config,
+						    BRU_PAD_SINK(0));
 		fmt->code = format->code;
 		break;
 	}
@@ -283,12 +294,16 @@ static int bru_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_bru *bru = to_bru(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	bru_try_format(bru, cfg, fmt->pad, &fmt->format, fmt->which);
+	config = vsp1_entity_get_pad_config(&bru->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	bru_try_format(bru, config, fmt->pad, &fmt->format);
 
-	format = vsp1_entity_get_pad_format(&bru->entity, cfg, fmt->pad,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&bru->entity, config, fmt->pad);
 	*format = fmt->format;
 
 	/* Reset the compose rectangle */
@@ -307,8 +322,8 @@ static int bru_set_format(struct v4l2_subdev *subdev,
 		unsigned int i;
 
 		for (i = 0; i <= bru->entity.source_pad; ++i) {
-			format = vsp1_entity_get_pad_format(&bru->entity, cfg,
-							    i, fmt->which);
+			format = vsp1_entity_get_pad_format(&bru->entity,
+							    config, i);
 			format->code = fmt->format.code;
 		}
 	}
@@ -347,6 +362,7 @@ static int bru_set_selection(struct v4l2_subdev *subdev,
 			     struct v4l2_subdev_selection *sel)
 {
 	struct vsp1_bru *bru = to_bru(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 	struct v4l2_rect *compose;
 
@@ -356,19 +372,22 @@ static int bru_set_selection(struct v4l2_subdev *subdev,
 	if (sel->target != V4L2_SEL_TGT_COMPOSE)
 		return -EINVAL;
 
+	config = vsp1_entity_get_pad_config(&bru->entity, cfg, sel->which);
+	if (!config)
+		return -EINVAL;
+
 	/* The compose rectangle top left corner must be inside the output
 	 * frame.
 	 */
-	format = vsp1_entity_get_pad_format(&bru->entity, cfg,
-					    bru->entity.source_pad, sel->which);
+	format = vsp1_entity_get_pad_format(&bru->entity, config,
+					    bru->entity.source_pad);
 	sel->r.left = clamp_t(unsigned int, sel->r.left, 0, format->width - 1);
 	sel->r.top = clamp_t(unsigned int, sel->r.top, 0, format->height - 1);
 
 	/* Scaling isn't supported, the compose rectangle size must be identical
 	 * to the sink format size.
 	 */
-	format = vsp1_entity_get_pad_format(&bru->entity, cfg, sel->pad,
-					    sel->which);
+	format = vsp1_entity_get_pad_format(&bru->entity, config, sel->pad);
 	sel->r.width = format->width;
 	sel->r.height = format->height;
 
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 3bd55d5ca739..57b746f81bc7 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -46,21 +46,48 @@ void vsp1_entity_route_setup(struct vsp1_entity *source)
  * V4L2 Subdevice Operations
  */
 
-struct v4l2_mbus_framefmt *
-vsp1_entity_get_pad_format(struct vsp1_entity *entity,
+/**
+ * vsp1_entity_get_pad_config - Get the pad configuration for an entity
+ * @entity: the entity
+ * @cfg: the TRY pad configuration
+ * @which: configuration selector (ACTIVE or TRY)
+ *
+ * Return the pad configuration requested by the which argument. The TRY
+ * configuration is passed explicitly to the function through the cfg argument
+ * and simply returned when requested. The ACTIVE configuration comes from the
+ * entity structure.
+ */
+struct v4l2_subdev_pad_config *
+vsp1_entity_get_pad_config(struct vsp1_entity *entity,
 			   struct v4l2_subdev_pad_config *cfg,
-			   unsigned int pad, u32 which)
+			   enum v4l2_subdev_format_whence which)
 {
 	switch (which) {
-	case V4L2_SUBDEV_FORMAT_TRY:
-		return v4l2_subdev_get_try_format(&entity->subdev, cfg, pad);
 	case V4L2_SUBDEV_FORMAT_ACTIVE:
-		return &entity->formats[pad];
+		return entity->config;
+	case V4L2_SUBDEV_FORMAT_TRY:
 	default:
-		return NULL;
+		return cfg;
 	}
 }
 
+/**
+ * vsp1_entity_get_pad_format - Get a pad format from storage for an entity
+ * @entity: the entity
+ * @cfg: the configuration storage
+ * @pad: the pad number
+ *
+ * Return the format stored in the given configuration for an entity's pad. The
+ * configuration can be an ACTIVE or TRY configuration.
+ */
+struct v4l2_mbus_framefmt *
+vsp1_entity_get_pad_format(struct vsp1_entity *entity,
+			   struct v4l2_subdev_pad_config *cfg,
+			   unsigned int pad)
+{
+	return v4l2_subdev_get_try_format(&entity->subdev, cfg, pad);
+}
+
 /*
  * vsp1_entity_init_cfg - Initialize formats on all pads
  * @subdev: V4L2 subdevice
@@ -169,19 +196,12 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 	entity->vsp1 = vsp1;
 	entity->source_pad = num_pads - 1;
 
-	/* Allocate formats and pads. */
-	entity->formats = devm_kzalloc(vsp1->dev,
-				       num_pads * sizeof(*entity->formats),
-				       GFP_KERNEL);
-	if (entity->formats == NULL)
-		return -ENOMEM;
-
+	/* Allocate and initialize pads. */
 	entity->pads = devm_kzalloc(vsp1->dev, num_pads * sizeof(*entity->pads),
 				    GFP_KERNEL);
 	if (entity->pads == NULL)
 		return -ENOMEM;
 
-	/* Initialize pads. */
 	for (i = 0; i < num_pads - 1; ++i)
 		entity->pads[i].flags = MEDIA_PAD_FL_SINK;
 
@@ -206,6 +226,15 @@ int vsp1_entity_init(struct vsp1_device *vsp1, struct vsp1_entity *entity,
 
 	vsp1_entity_init_cfg(subdev, NULL);
 
+	/* Allocate the pad configuration to store formats and selection
+	 * rectangles.
+	 */
+	entity->config = v4l2_subdev_alloc_pad_config(&entity->subdev);
+	if (entity->config == NULL) {
+		media_entity_cleanup(&entity->subdev.entity);
+		return -ENOMEM;
+	}
+
 	return 0;
 }
 
@@ -215,5 +244,6 @@ void vsp1_entity_destroy(struct vsp1_entity *entity)
 		entity->ops->destroy(entity);
 	if (entity->subdev.ctrl_handler)
 		v4l2_ctrl_handler_free(entity->subdev.ctrl_handler);
+	v4l2_subdev_free_pad_config(entity->config);
 	media_entity_cleanup(&entity->subdev.entity);
 }
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index b608eef9700c..6fd53bc80aa2 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -84,7 +84,7 @@ struct vsp1_entity {
 	unsigned int sink_pad;
 
 	struct v4l2_subdev subdev;
-	struct v4l2_mbus_framefmt *formats;
+	struct v4l2_subdev_pad_config *config;
 };
 
 static inline struct vsp1_entity *to_vsp1_entity(struct v4l2_subdev *subdev)
@@ -103,10 +103,14 @@ int vsp1_entity_link_setup(struct media_entity *entity,
 			   const struct media_pad *local,
 			   const struct media_pad *remote, u32 flags);
 
+struct v4l2_subdev_pad_config *
+vsp1_entity_get_pad_config(struct vsp1_entity *entity,
+			   struct v4l2_subdev_pad_config *cfg,
+			   enum v4l2_subdev_format_whence which);
 struct v4l2_mbus_framefmt *
 vsp1_entity_get_pad_format(struct vsp1_entity *entity,
 			   struct v4l2_subdev_pad_config *cfg,
-			   unsigned int pad, u32 which);
+			   unsigned int pad);
 int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
 			 struct v4l2_subdev_pad_config *cfg);
 
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index f20293d6587b..6565ef62c376 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -77,10 +77,14 @@ static int hsit_enum_frame_size(struct v4l2_subdev *subdev,
 				struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_hsit *hsit = to_hsit(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	format = vsp1_entity_get_pad_format(&hsit->entity, cfg, fse->pad,
-					    fse->which);
+	config = vsp1_entity_get_pad_config(&hsit->entity, cfg, fse->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(&hsit->entity, config, fse->pad);
 
 	if (fse->index || fse->code != format->code)
 		return -EINVAL;
@@ -108,9 +112,14 @@ static int hsit_get_format(struct v4l2_subdev *subdev,
 			   struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_hsit *hsit = to_hsit(subdev);
+	struct v4l2_subdev_pad_config *config;
+
+	config = vsp1_entity_get_pad_config(&hsit->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
 
-	fmt->format = *vsp1_entity_get_pad_format(&hsit->entity, cfg, fmt->pad,
-						  fmt->which);
+	fmt->format = *vsp1_entity_get_pad_format(&hsit->entity, config,
+						  fmt->pad);
 
 	return 0;
 }
@@ -120,10 +129,14 @@ static int hsit_set_format(struct v4l2_subdev *subdev,
 			   struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_hsit *hsit = to_hsit(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	format = vsp1_entity_get_pad_format(&hsit->entity, cfg, fmt->pad,
-					    fmt->which);
+	config = vsp1_entity_get_pad_config(&hsit->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(&hsit->entity, config, fmt->pad);
 
 	if (fmt->pad == HSIT_PAD_SOURCE) {
 		/* The HST and HSI output format code and resolution can't be
@@ -145,8 +158,8 @@ static int hsit_set_format(struct v4l2_subdev *subdev,
 	fmt->format = *format;
 
 	/* Propagate the format to the source pad. */
-	format = vsp1_entity_get_pad_format(&hsit->entity, cfg, HSIT_PAD_SOURCE,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&hsit->entity, config,
+					    HSIT_PAD_SOURCE);
 	*format = fmt->format;
 	format->code = hsit->inverse ? MEDIA_BUS_FMT_ARGB8888_1X32
 		     : MEDIA_BUS_FMT_AHSV8888_1X32;
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index f7e51ddb8321..9ab40d843578 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -48,7 +48,8 @@ static int lif_s_stream(struct v4l2_subdev *subdev, int enable)
 		return 0;
 	}
 
-	format = &lif->entity.formats[LIF_PAD_SOURCE];
+	format = vsp1_entity_get_pad_format(&lif->entity, lif->entity.config,
+					    LIF_PAD_SOURCE);
 
 	obth = min(obth, (format->width + 1) / 2 * format->height - 4);
 
@@ -84,6 +85,7 @@ static int lif_enum_mbus_code(struct v4l2_subdev *subdev,
 
 		code->code = codes[code->index];
 	} else {
+		struct v4l2_subdev_pad_config *config;
 		struct v4l2_mbus_framefmt *format;
 
 		/* The LIF can't perform format conversion, the sink format is
@@ -92,8 +94,13 @@ static int lif_enum_mbus_code(struct v4l2_subdev *subdev,
 		if (code->index)
 			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&lif->entity, cfg,
-						    LIF_PAD_SINK, code->which);
+		config = vsp1_entity_get_pad_config(&lif->entity, cfg,
+						    code->which);
+		if (!config)
+			return -EINVAL;
+
+		format = vsp1_entity_get_pad_format(&lif->entity, config,
+						    LIF_PAD_SINK);
 		code->code = format->code;
 	}
 
@@ -105,10 +112,14 @@ static int lif_enum_frame_size(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_lif *lif = to_lif(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	format = vsp1_entity_get_pad_format(&lif->entity, cfg, LIF_PAD_SINK,
-					    fse->which);
+	config = vsp1_entity_get_pad_config(&lif->entity, cfg, fse->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(&lif->entity, config, LIF_PAD_SINK);
 
 	if (fse->index || fse->code != format->code)
 		return -EINVAL;
@@ -133,9 +144,14 @@ static int lif_get_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lif *lif = to_lif(subdev);
+	struct v4l2_subdev_pad_config *config;
 
-	fmt->format = *vsp1_entity_get_pad_format(&lif->entity, cfg, fmt->pad,
-						  fmt->which);
+	config = vsp1_entity_get_pad_config(&lif->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	fmt->format = *vsp1_entity_get_pad_format(&lif->entity, config,
+						  fmt->pad);
 
 	return 0;
 }
@@ -145,15 +161,19 @@ static int lif_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lif *lif = to_lif(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
+	config = vsp1_entity_get_pad_config(&lif->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
 	/* Default to YUV if the requested format is not supported. */
 	if (fmt->format.code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
 	    fmt->format.code != MEDIA_BUS_FMT_AYUV8_1X32)
 		fmt->format.code = MEDIA_BUS_FMT_AYUV8_1X32;
 
-	format = vsp1_entity_get_pad_format(&lif->entity, cfg, fmt->pad,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&lif->entity, config, fmt->pad);
 
 	if (fmt->pad == LIF_PAD_SOURCE) {
 		/* The LIF source format is always identical to its sink
@@ -174,8 +194,8 @@ static int lif_set_format(struct v4l2_subdev *subdev,
 	fmt->format = *format;
 
 	/* Propagate the format to the source pad. */
-	format = vsp1_entity_get_pad_format(&lif->entity, cfg, LIF_PAD_SOURCE,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&lif->entity, config,
+					    LIF_PAD_SOURCE);
 	*format = fmt->format;
 
 	return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 9448c4c376aa..9510424113b2 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -86,7 +86,6 @@ static int lut_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
 	struct vsp1_lut *lut = to_lut(subdev);
-	struct v4l2_mbus_framefmt *format;
 
 	if (code->pad == LUT_PAD_SINK) {
 		if (code->index >= ARRAY_SIZE(codes))
@@ -94,14 +93,22 @@ static int lut_enum_mbus_code(struct v4l2_subdev *subdev,
 
 		code->code = codes[code->index];
 	} else {
+		struct v4l2_subdev_pad_config *config;
+		struct v4l2_mbus_framefmt *format;
+
 		/* The LUT can't perform format conversion, the sink format is
 		 * always identical to the source format.
 		 */
 		if (code->index)
 			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&lut->entity, cfg,
-						    LUT_PAD_SINK, code->which);
+		config = vsp1_entity_get_pad_config(&lut->entity, cfg,
+						    code->which);
+		if (!config)
+			return -EINVAL;
+
+		format = vsp1_entity_get_pad_format(&lut->entity, config,
+						    LUT_PAD_SINK);
 		code->code = format->code;
 	}
 
@@ -113,10 +120,14 @@ static int lut_enum_frame_size(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_lut *lut = to_lut(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	format = vsp1_entity_get_pad_format(&lut->entity, cfg,
-					    fse->pad, fse->which);
+	config = vsp1_entity_get_pad_config(&lut->entity, cfg, fse->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(&lut->entity, config, fse->pad);
 
 	if (fse->index || fse->code != format->code)
 		return -EINVAL;
@@ -144,9 +155,14 @@ static int lut_get_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lut *lut = to_lut(subdev);
+	struct v4l2_subdev_pad_config *config;
+
+	config = vsp1_entity_get_pad_config(&lut->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
 
-	fmt->format = *vsp1_entity_get_pad_format(&lut->entity, cfg, fmt->pad,
-						  fmt->which);
+	fmt->format = *vsp1_entity_get_pad_format(&lut->entity, config,
+						  fmt->pad);
 
 	return 0;
 }
@@ -156,16 +172,20 @@ static int lut_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_lut *lut = to_lut(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
+	config = vsp1_entity_get_pad_config(&lut->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
 	/* Default to YUV if the requested format is not supported. */
 	if (fmt->format.code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
 	    fmt->format.code != MEDIA_BUS_FMT_AHSV8888_1X32 &&
 	    fmt->format.code != MEDIA_BUS_FMT_AYUV8_1X32)
 		fmt->format.code = MEDIA_BUS_FMT_AYUV8_1X32;
 
-	format = vsp1_entity_get_pad_format(&lut->entity, cfg, fmt->pad,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&lut->entity, config, fmt->pad);
 
 	if (fmt->pad == LUT_PAD_SOURCE) {
 		/* The LUT output format can't be modified. */
@@ -183,8 +203,8 @@ static int lut_set_format(struct v4l2_subdev *subdev,
 	fmt->format = *format;
 
 	/* Propagate the format to the source pad. */
-	format = vsp1_entity_get_pad_format(&lut->entity, cfg, LUT_PAD_SOURCE,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&lut->entity, config,
+					    LUT_PAD_SOURCE);
 	*format = fmt->format;
 
 	return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 51542bccd450..3485ad0f61ed 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -42,6 +42,8 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	struct vsp1_rwpf *rpf = to_rwpf(subdev);
 	const struct vsp1_format_info *fmtinfo = rpf->fmtinfo;
 	const struct v4l2_pix_format_mplane *format = &rpf->format;
+	const struct v4l2_mbus_framefmt *source_format;
+	const struct v4l2_mbus_framefmt *sink_format;
 	const struct v4l2_rect *crop = &rpf->crop;
 	u32 pstride;
 	u32 infmt;
@@ -79,6 +81,13 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	vsp1_rpf_write(rpf, VI6_RPF_SRCM_PSTRIDE, pstride);
 
 	/* Format */
+	sink_format = vsp1_entity_get_pad_format(&rpf->entity,
+						 rpf->entity.config,
+						 RWPF_PAD_SINK);
+	source_format = vsp1_entity_get_pad_format(&rpf->entity,
+						   rpf->entity.config,
+						   RWPF_PAD_SOURCE);
+
 	infmt = VI6_RPF_INFMT_CIPM
 	      | (fmtinfo->hwfmt << VI6_RPF_INFMT_RDFMT_SHIFT);
 
@@ -87,8 +96,7 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	if (fmtinfo->swap_uv)
 		infmt |= VI6_RPF_INFMT_SPUVS;
 
-	if (rpf->entity.formats[RWPF_PAD_SINK].code !=
-	    rpf->entity.formats[RWPF_PAD_SOURCE].code)
+	if (sink_format->code != source_format->code)
 		infmt |= VI6_RPF_INFMT_CSC;
 
 	vsp1_rpf_write(rpf, VI6_RPF_INFMT, infmt);
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 38893ab06cd9..e5216d39723e 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -46,10 +46,14 @@ int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 			      struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, fse->pad,
-					    fse->which);
+	config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, fse->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(&rwpf->entity, config, fse->pad);
 
 	if (fse->index || fse->code != format->code)
 		return -EINVAL;
@@ -92,9 +96,14 @@ int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
 			 struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+	struct v4l2_subdev_pad_config *config;
 
-	fmt->format = *vsp1_entity_get_pad_format(&rwpf->entity, cfg, fmt->pad,
-						  fmt->which);
+	config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	fmt->format = *vsp1_entity_get_pad_format(&rwpf->entity, config,
+						  fmt->pad);
 
 	return 0;
 }
@@ -104,16 +113,20 @@ int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 			 struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 	struct v4l2_rect *crop;
 
+	config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
 	/* Default to YUV if the requested format is not supported. */
 	if (fmt->format.code != MEDIA_BUS_FMT_ARGB8888_1X32 &&
 	    fmt->format.code != MEDIA_BUS_FMT_AYUV8_1X32)
 		fmt->format.code = MEDIA_BUS_FMT_AYUV8_1X32;
 
-	format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, fmt->pad,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&rwpf->entity, config, fmt->pad);
 
 	if (fmt->pad == RWPF_PAD_SOURCE) {
 		/* The RWPF performs format conversion but can't scale, only the
@@ -142,8 +155,8 @@ int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 	crop->height = fmt->format.height;
 
 	/* Propagate the format to the source pad. */
-	format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, RWPF_PAD_SOURCE,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+					    RWPF_PAD_SOURCE);
 	*format = fmt->format;
 
 	return 0;
@@ -154,20 +167,25 @@ int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
 			    struct v4l2_subdev_selection *sel)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
 	/* Cropping is implemented on the sink pad. */
 	if (sel->pad != RWPF_PAD_SINK)
 		return -EINVAL;
 
+	config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, sel->which);
+	if (!config)
+		return -EINVAL;
+
 	switch (sel->target) {
 	case V4L2_SEL_TGT_CROP:
 		sel->r = *vsp1_rwpf_get_crop(rwpf, cfg, sel->which);
 		break;
 
 	case V4L2_SEL_TGT_CROP_BOUNDS:
-		format = vsp1_entity_get_pad_format(&rwpf->entity, cfg,
-						    RWPF_PAD_SINK, sel->which);
+		format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+						    RWPF_PAD_SINK);
 		sel->r.left = 0;
 		sel->r.top = 0;
 		sel->r.width = format->width;
@@ -186,6 +204,7 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 			    struct v4l2_subdev_selection *sel)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 	struct v4l2_rect *crop;
 
@@ -196,11 +215,15 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 	if (sel->target != V4L2_SEL_TGT_CROP)
 		return -EINVAL;
 
+	config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, sel->which);
+	if (!config)
+		return -EINVAL;
+
 	/* Make sure the crop rectangle is entirely contained in the image. The
 	 * WPF top and left offsets are limited to 255.
 	 */
-	format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, RWPF_PAD_SINK,
-					    sel->which);
+	format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+					    RWPF_PAD_SINK);
 
 	/* Restrict the crop rectangle coordinates to multiples of 2 to avoid
 	 * shifting the color plane.
@@ -227,8 +250,8 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 	*crop = sel->r;
 
 	/* Propagate the format to the source pad. */
-	format = vsp1_entity_get_pad_format(&rwpf->entity, cfg, RWPF_PAD_SOURCE,
-					    sel->which);
+	format = vsp1_entity_get_pad_format(&rwpf->entity, config,
+					    RWPF_PAD_SOURCE);
 	format->width = crop->width;
 	format->height = crop->height;
 
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 6eae5a493bc3..a67fa0c9fe3b 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -117,8 +117,10 @@ static int sru_s_stream(struct v4l2_subdev *subdev, int enable)
 	if (!enable)
 		return 0;
 
-	input = &sru->entity.formats[SRU_PAD_SINK];
-	output = &sru->entity.formats[SRU_PAD_SOURCE];
+	input = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
+					   SRU_PAD_SINK);
+	output = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
+					    SRU_PAD_SOURCE);
 
 	if (input->code == MEDIA_BUS_FMT_ARGB8888_1X32)
 		ctrl0 = VI6_SRU_CTRL0_PARAM2 | VI6_SRU_CTRL0_PARAM3
@@ -153,7 +155,6 @@ static int sru_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
 	struct vsp1_sru *sru = to_sru(subdev);
-	struct v4l2_mbus_framefmt *format;
 
 	if (code->pad == SRU_PAD_SINK) {
 		if (code->index >= ARRAY_SIZE(codes))
@@ -161,14 +162,22 @@ static int sru_enum_mbus_code(struct v4l2_subdev *subdev,
 
 		code->code = codes[code->index];
 	} else {
+		struct v4l2_subdev_pad_config *config;
+		struct v4l2_mbus_framefmt *format;
+
 		/* The SRU can't perform format conversion, the sink format is
 		 * always identical to the source format.
 		 */
 		if (code->index)
 			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&sru->entity, cfg,
-						    SRU_PAD_SINK, code->which);
+		config = vsp1_entity_get_pad_config(&sru->entity, cfg,
+						    code->which);
+		if (!config)
+			return -EINVAL;
+
+		format = vsp1_entity_get_pad_format(&sru->entity, config,
+						    SRU_PAD_SINK);
 		code->code = format->code;
 	}
 
@@ -180,10 +189,14 @@ static int sru_enum_frame_size(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_sru *sru = to_sru(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	format = vsp1_entity_get_pad_format(&sru->entity, cfg,
-					    SRU_PAD_SINK, fse->which);
+	config = vsp1_entity_get_pad_config(&sru->entity, cfg, fse->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(&sru->entity, config, SRU_PAD_SINK);
 
 	if (fse->index || fse->code != format->code)
 		return -EINVAL;
@@ -214,17 +227,21 @@ static int sru_get_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_sru *sru = to_sru(subdev);
+	struct v4l2_subdev_pad_config *config;
+
+	config = vsp1_entity_get_pad_config(&sru->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
 
-	fmt->format = *vsp1_entity_get_pad_format(&sru->entity, cfg, fmt->pad,
-						  fmt->which);
+	fmt->format = *vsp1_entity_get_pad_format(&sru->entity, config,
+						  fmt->pad);
 
 	return 0;
 }
 
 static void sru_try_format(struct vsp1_sru *sru,
-			   struct v4l2_subdev_pad_config *cfg,
-			   unsigned int pad, struct v4l2_mbus_framefmt *fmt,
-			   enum v4l2_subdev_format_whence which)
+			   struct v4l2_subdev_pad_config *config,
+			   unsigned int pad, struct v4l2_mbus_framefmt *fmt)
 {
 	struct v4l2_mbus_framefmt *format;
 	unsigned int input_area;
@@ -243,8 +260,8 @@ static void sru_try_format(struct vsp1_sru *sru,
 
 	case SRU_PAD_SOURCE:
 		/* The SRU can't perform format conversion. */
-		format = vsp1_entity_get_pad_format(&sru->entity, cfg,
-						    SRU_PAD_SINK, which);
+		format = vsp1_entity_get_pad_format(&sru->entity, config,
+						    SRU_PAD_SINK);
 		fmt->code = format->code;
 
 		/* We can upscale by 2 in both direction, but not independently.
@@ -278,21 +295,25 @@ static int sru_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_sru *sru = to_sru(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	sru_try_format(sru, cfg, fmt->pad, &fmt->format, fmt->which);
+	config = vsp1_entity_get_pad_config(&sru->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	sru_try_format(sru, config, fmt->pad, &fmt->format);
 
-	format = vsp1_entity_get_pad_format(&sru->entity, cfg, fmt->pad,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&sru->entity, config, fmt->pad);
 	*format = fmt->format;
 
 	if (fmt->pad == SRU_PAD_SINK) {
 		/* Propagate the format to the source pad. */
-		format = vsp1_entity_get_pad_format(&sru->entity, cfg,
-						    SRU_PAD_SOURCE, fmt->which);
+		format = vsp1_entity_get_pad_format(&sru->entity, config,
+						    SRU_PAD_SOURCE);
 		*format = fmt->format;
 
-		sru_try_format(sru, cfg, SRU_PAD_SOURCE, format, fmt->which);
+		sru_try_format(sru, config, SRU_PAD_SOURCE, format);
 	}
 
 	return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 6b6d92c5c629..c2209823140e 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -120,8 +120,10 @@ static int uds_s_stream(struct v4l2_subdev *subdev, int enable)
 	if (!enable)
 		return 0;
 
-	input = &uds->entity.formats[UDS_PAD_SINK];
-	output = &uds->entity.formats[UDS_PAD_SOURCE];
+	input = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
+					   UDS_PAD_SINK);
+	output = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
+					    UDS_PAD_SOURCE);
 
 	hscale = uds_compute_ratio(input->width, output->width);
 	vscale = uds_compute_ratio(input->height, output->height);
@@ -178,16 +180,22 @@ static int uds_enum_mbus_code(struct v4l2_subdev *subdev,
 
 		code->code = codes[code->index];
 	} else {
+		struct v4l2_subdev_pad_config *config;
 		struct v4l2_mbus_framefmt *format;
 
+		config = vsp1_entity_get_pad_config(&uds->entity, cfg,
+						    code->which);
+		if (!config)
+			return -EINVAL;
+
 		/* The UDS can't perform format conversion, the sink format is
 		 * always identical to the source format.
 		 */
 		if (code->index)
 			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&uds->entity, cfg,
-						    UDS_PAD_SINK, code->which);
+		format = vsp1_entity_get_pad_format(&uds->entity, config,
+						    UDS_PAD_SINK);
 		code->code = format->code;
 	}
 
@@ -199,10 +207,15 @@ static int uds_enum_frame_size(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_uds *uds = to_uds(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	format = vsp1_entity_get_pad_format(&uds->entity, cfg,
-					    UDS_PAD_SINK, fse->which);
+	config = vsp1_entity_get_pad_config(&uds->entity, cfg, fse->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(&uds->entity, config,
+					    UDS_PAD_SINK);
 
 	if (fse->index || fse->code != format->code)
 		return -EINVAL;
@@ -227,17 +240,21 @@ static int uds_get_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_uds *uds = to_uds(subdev);
+	struct v4l2_subdev_pad_config *config;
 
-	fmt->format = *vsp1_entity_get_pad_format(&uds->entity, cfg, fmt->pad,
-						  fmt->which);
+	config = vsp1_entity_get_pad_config(&uds->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	fmt->format = *vsp1_entity_get_pad_format(&uds->entity, config,
+						  fmt->pad);
 
 	return 0;
 }
 
 static void uds_try_format(struct vsp1_uds *uds,
-			   struct v4l2_subdev_pad_config *cfg,
-			   unsigned int pad, struct v4l2_mbus_framefmt *fmt,
-			   enum v4l2_subdev_format_whence which)
+			   struct v4l2_subdev_pad_config *config,
+			   unsigned int pad, struct v4l2_mbus_framefmt *fmt)
 {
 	struct v4l2_mbus_framefmt *format;
 	unsigned int minimum;
@@ -256,8 +273,8 @@ static void uds_try_format(struct vsp1_uds *uds,
 
 	case UDS_PAD_SOURCE:
 		/* The UDS scales but can't perform format conversion. */
-		format = vsp1_entity_get_pad_format(&uds->entity, cfg,
-						    UDS_PAD_SINK, which);
+		format = vsp1_entity_get_pad_format(&uds->entity, config,
+						    UDS_PAD_SINK);
 		fmt->code = format->code;
 
 		uds_output_limits(format->width, &minimum, &maximum);
@@ -276,21 +293,25 @@ static int uds_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_uds *uds = to_uds(subdev);
+	struct v4l2_subdev_pad_config *config;
 	struct v4l2_mbus_framefmt *format;
 
-	uds_try_format(uds, cfg, fmt->pad, &fmt->format, fmt->which);
+	config = vsp1_entity_get_pad_config(&uds->entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	uds_try_format(uds, config, fmt->pad, &fmt->format);
 
-	format = vsp1_entity_get_pad_format(&uds->entity, cfg, fmt->pad,
-					    fmt->which);
+	format = vsp1_entity_get_pad_format(&uds->entity, config, fmt->pad);
 	*format = fmt->format;
 
 	if (fmt->pad == UDS_PAD_SINK) {
 		/* Propagate the format to the source pad. */
-		format = vsp1_entity_get_pad_format(&uds->entity, cfg,
-						    UDS_PAD_SOURCE, fmt->which);
+		format = vsp1_entity_get_pad_format(&uds->entity, config,
+						    UDS_PAD_SOURCE);
 		*format = fmt->format;
 
-		uds_try_format(uds, cfg, UDS_PAD_SOURCE, format, fmt->which);
+		uds_try_format(uds, config, UDS_PAD_SOURCE, format);
 	}
 
 	return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 7f6a7cb00706..254f90a85162 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -42,6 +42,8 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
 	struct vsp1_rwpf *wpf = to_rwpf(subdev);
 	struct vsp1_device *vsp1 = wpf->entity.vsp1;
+	const struct v4l2_mbus_framefmt *source_format;
+	const struct v4l2_mbus_framefmt *sink_format;
 	const struct v4l2_rect *crop = &wpf->crop;
 	unsigned int i;
 	u32 srcrpf = 0;
@@ -94,6 +96,13 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 		       (crop->height << VI6_WPF_SZCLIP_SIZE_SHIFT));
 
 	/* Format */
+	sink_format = vsp1_entity_get_pad_format(&wpf->entity,
+						 wpf->entity.config,
+						 RWPF_PAD_SINK);
+	source_format = vsp1_entity_get_pad_format(&wpf->entity,
+						   wpf->entity.config,
+						   RWPF_PAD_SOURCE);
+
 	if (!pipe->lif) {
 		const struct vsp1_format_info *fmtinfo = wpf->fmtinfo;
 
@@ -109,8 +118,7 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 		vsp1_wpf_write(wpf, VI6_WPF_DSWAP, fmtinfo->swap);
 	}
 
-	if (wpf->entity.formats[RWPF_PAD_SINK].code !=
-	    wpf->entity.formats[RWPF_PAD_SOURCE].code)
+	if (sink_format->code != source_format->code)
 		outfmt |= VI6_WPF_OUTFMT_CSC;
 
 	outfmt |= wpf->alpha << VI6_WPF_OUTFMT_PDV_SHIFT;
-- 
2.7.3

[PATCH v2 31/54] v4l: vsp1: Store active selection rectangles in a pad config structure

[Laurent Pinchart]

Use the pad config structure part of the vsp1_entity to store all active
pad selection rectangles. This generalizes the code to operate on pad
config structures.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 24 +++++++++++-------------
 drivers/media/platform/vsp1/vsp1_bru.h    |  1 -
 drivers/media/platform/vsp1/vsp1_drm.c    |  5 ++---
 drivers/media/platform/vsp1/vsp1_entity.c |  8 ++++++++
 drivers/media/platform/vsp1/vsp1_entity.h |  4 ++++
 drivers/media/platform/vsp1/vsp1_rpf.c    | 20 +++++++++++++++++---
 drivers/media/platform/vsp1/vsp1_rwpf.c   | 22 +++++++---------------
 drivers/media/platform/vsp1/vsp1_rwpf.h   |  8 +++-----
 drivers/media/platform/vsp1/vsp1_video.c  | 13 +++----------
 drivers/media/platform/vsp1/vsp1_wpf.c    |  4 +++-
 10 files changed, 58 insertions(+), 51 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 1691d13131a9..d3b63056450a 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -231,17 +231,9 @@ static int bru_enum_frame_size(struct v4l2_subdev *subdev,
 
 static struct v4l2_rect *bru_get_compose(struct vsp1_bru *bru,
 					 struct v4l2_subdev_pad_config *cfg,
-					 unsigned int pad, u32 which)
+					 unsigned int pad)
 {
-	switch (which) {
-	case V4L2_SUBDEV_FORMAT_TRY:
-		return v4l2_subdev_get_try_compose(&bru->entity.subdev, cfg,
-						   pad);
-	case V4L2_SUBDEV_FORMAT_ACTIVE:
-		return &bru->inputs[pad].compose;
-	default:
-		return NULL;
-	}
+	return v4l2_subdev_get_try_compose(&bru->entity.subdev, cfg, pad);
 }
 
 static int bru_get_format(struct v4l2_subdev *subdev,
@@ -310,7 +302,7 @@ static int bru_set_format(struct v4l2_subdev *subdev,
 	if (fmt->pad != bru->entity.source_pad) {
 		struct v4l2_rect *compose;
 
-		compose = bru_get_compose(bru, cfg, fmt->pad, fmt->which);
+		compose = bru_get_compose(bru, config, fmt->pad);
 		compose->left = 0;
 		compose->top = 0;
 		compose->width = format->width;
@@ -336,6 +328,7 @@ static int bru_get_selection(struct v4l2_subdev *subdev,
 			     struct v4l2_subdev_selection *sel)
 {
 	struct vsp1_bru *bru = to_bru(subdev);
+	struct v4l2_subdev_pad_config *config;
 
 	if (sel->pad == bru->entity.source_pad)
 		return -EINVAL;
@@ -349,7 +342,12 @@ static int bru_get_selection(struct v4l2_subdev *subdev,
 		return 0;
 
 	case V4L2_SEL_TGT_COMPOSE:
-		sel->r = *bru_get_compose(bru, cfg, sel->pad, sel->which);
+		config = vsp1_entity_get_pad_config(&bru->entity, cfg,
+						    sel->which);
+		if (!config)
+			return -EINVAL;
+
+		sel->r = *bru_get_compose(bru, config, sel->pad);
 		return 0;
 
 	default:
@@ -391,7 +389,7 @@ static int bru_set_selection(struct v4l2_subdev *subdev,
 	sel->r.width = format->width;
 	sel->r.height = format->height;
 
-	compose = bru_get_compose(bru, cfg, sel->pad, sel->which);
+	compose = bru_get_compose(bru, config, sel->pad);
 	*compose = sel->r;
 
 	return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_bru.h b/drivers/media/platform/vsp1/vsp1_bru.h
index 4e7d2e79b940..828a3fcadea8 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.h
+++ b/drivers/media/platform/vsp1/vsp1_bru.h
@@ -31,7 +31,6 @@ struct vsp1_bru {
 
 	struct {
 		struct vsp1_rwpf *rpf;
-		struct v4l2_rect compose;
 	} inputs[VSP1_MAX_RPF];
 
 	u32 bgcolor;
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index a73018c9e8b5..acbf36d315b9 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -410,9 +410,8 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 		__func__, sel.r.left, sel.r.top, sel.r.width, sel.r.height,
 		sel.pad);
 
-	/* Store the compose rectangle coordinates in the RPF. */
-	rpf->location.left = dst->left;
-	rpf->location.top = dst->top;
+	/* Store the BRU input pad number in the RPF. */
+	rpf->bru_input = rpf->entity.index;
 
 	/* Cache the memory buffer address but don't apply the values to the
 	 * hardware as the crop offsets haven't been computed yet.
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 57b746f81bc7..37519acd1691 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -88,6 +88,14 @@ vsp1_entity_get_pad_format(struct vsp1_entity *entity,
 	return v4l2_subdev_get_try_format(&entity->subdev, cfg, pad);
 }
 
+struct v4l2_rect *
+vsp1_entity_get_pad_compose(struct vsp1_entity *entity,
+			    struct v4l2_subdev_pad_config *cfg,
+			    unsigned int pad)
+{
+	return v4l2_subdev_get_try_compose(&entity->subdev, cfg, pad);
+}
+
 /*
  * vsp1_entity_init_cfg - Initialize formats on all pads
  * @subdev: V4L2 subdevice
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 6fd53bc80aa2..1935ae9289ba 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -111,6 +111,10 @@ struct v4l2_mbus_framefmt *
 vsp1_entity_get_pad_format(struct vsp1_entity *entity,
 			   struct v4l2_subdev_pad_config *cfg,
 			   unsigned int pad);
+struct v4l2_rect *
+vsp1_entity_get_pad_compose(struct vsp1_entity *entity,
+			    struct v4l2_subdev_pad_config *cfg,
+			    unsigned int pad);
 int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
 			 struct v4l2_subdev_pad_config *cfg);
 
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 3485ad0f61ed..198487952418 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -44,7 +44,9 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	const struct v4l2_pix_format_mplane *format = &rpf->format;
 	const struct v4l2_mbus_framefmt *source_format;
 	const struct v4l2_mbus_framefmt *sink_format;
-	const struct v4l2_rect *crop = &rpf->crop;
+	const struct v4l2_rect *crop;
+	unsigned int left = 0;
+	unsigned int top = 0;
 	u32 pstride;
 	u32 infmt;
 
@@ -57,6 +59,8 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	 * left corner in the plane buffer. Only two offsets are needed, as
 	 * planes 2 and 3 always have identical strides.
 	 */
+	crop = vsp1_rwpf_get_crop(rpf, rpf->entity.config);
+
 	vsp1_rpf_write(rpf, VI6_RPF_SRC_BSIZE,
 		       (crop->width << VI6_RPF_SRC_BSIZE_BHSIZE_SHIFT) |
 		       (crop->height << VI6_RPF_SRC_BSIZE_BVSIZE_SHIFT));
@@ -103,9 +107,19 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	vsp1_rpf_write(rpf, VI6_RPF_DSWAP, fmtinfo->swap);
 
 	/* Output location */
+	if (pipe->bru) {
+		const struct v4l2_rect *compose;
+
+		compose = vsp1_entity_get_pad_compose(pipe->bru,
+						      pipe->bru->config,
+						      rpf->bru_input);
+		left = compose->left;
+		top = compose->top;
+	}
+
 	vsp1_rpf_write(rpf, VI6_RPF_LOC,
-		       (rpf->location.left << VI6_RPF_LOC_HCOORD_SHIFT) |
-		       (rpf->location.top << VI6_RPF_LOC_VCOORD_SHIFT));
+		       (left << VI6_RPF_LOC_HCOORD_SHIFT) |
+		       (top << VI6_RPF_LOC_VCOORD_SHIFT));
 
 	/* Use the alpha channel (extended to 8 bits) when available or an
 	 * alpha value set through the V4L2_CID_ALPHA_COMPONENT control
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index e5216d39723e..0c5ad023adfb 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -76,19 +76,11 @@ int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static struct v4l2_rect *
-vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf, struct v4l2_subdev_pad_config *cfg,
-		   u32 which)
+struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
+				     struct v4l2_subdev_pad_config *config)
 {
-	switch (which) {
-	case V4L2_SUBDEV_FORMAT_TRY:
-		return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, cfg,
-						RWPF_PAD_SINK);
-	case V4L2_SUBDEV_FORMAT_ACTIVE:
-		return &rwpf->crop;
-	default:
-		return NULL;
-	}
+	return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, config,
+					RWPF_PAD_SINK);
 }
 
 int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
@@ -148,7 +140,7 @@ int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 	fmt->format = *format;
 
 	/* Update the sink crop rectangle. */
-	crop = vsp1_rwpf_get_crop(rwpf, cfg, fmt->which);
+	crop = vsp1_rwpf_get_crop(rwpf, config);
 	crop->left = 0;
 	crop->top = 0;
 	crop->width = fmt->format.width;
@@ -180,7 +172,7 @@ int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
 
 	switch (sel->target) {
 	case V4L2_SEL_TGT_CROP:
-		sel->r = *vsp1_rwpf_get_crop(rwpf, cfg, sel->which);
+		sel->r = *vsp1_rwpf_get_crop(rwpf, config);
 		break;
 
 	case V4L2_SEL_TGT_CROP_BOUNDS:
@@ -246,7 +238,7 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 	sel->r.height = min_t(unsigned int, sel->r.height,
 			      format->height - sel->r.top);
 
-	crop = vsp1_rwpf_get_crop(rwpf, cfg, sel->which);
+	crop = vsp1_rwpf_get_crop(rwpf, config);
 	*crop = sel->r;
 
 	/* Propagate the format to the source pad. */
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index e8ca9b6ee689..4ebfab61e0ef 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -43,11 +43,7 @@ struct vsp1_rwpf {
 
 	struct v4l2_pix_format_mplane format;
 	const struct vsp1_format_info *fmtinfo;
-	struct {
-		unsigned int left;
-		unsigned int top;
-	} location;
-	struct v4l2_rect crop;
+	unsigned int bru_input;
 
 	unsigned int alpha;
 
@@ -91,6 +87,8 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 			    struct v4l2_subdev_pad_config *cfg,
 			    struct v4l2_subdev_selection *sel);
 
+struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
+				     struct v4l2_subdev_pad_config *config);
 /**
  * vsp1_rwpf_set_memory - Configure DMA addresses for a [RW]PF
  * @rwpf: the [RW]PF instance
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 102977ae1daa..d4a092c8ece3 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -182,9 +182,6 @@ static int vsp1_video_pipeline_validate_branch(struct vsp1_pipeline *pipe,
 	bool bru_found = false;
 	int ret;
 
-	input->location.left = 0;
-	input->location.top = 0;
-
 	ret = media_entity_enum_init(&ent_enum, &input->entity.vsp1->media_dev);
 	if (ret < 0)
 		return ret;
@@ -206,18 +203,14 @@ static int vsp1_video_pipeline_validate_branch(struct vsp1_pipeline *pipe,
 		entity = to_vsp1_entity(
 			media_entity_to_v4l2_subdev(pad->entity));
 
-		/* A BRU is present in the pipeline, store the compose rectangle
-		 * location in the input RPF for use when configuring the RPF.
+		/* A BRU is present in the pipeline, store the BRU input pad
+		 * number in the input RPF for use when configuring the RPF.
 		 */
 		if (entity->type == VSP1_ENTITY_BRU) {
 			struct vsp1_bru *bru = to_bru(&entity->subdev);
-			struct v4l2_rect *rect =
-				&bru->inputs[pad->index].compose;
 
 			bru->inputs[pad->index].rpf = input;
-
-			input->location.left = rect->left;
-			input->location.top = rect->top;
+			input->bru_input = pad->index;
 
 			bru_found = true;
 		}
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 254f90a85162..962b43b7f99e 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -44,7 +44,7 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	struct vsp1_device *vsp1 = wpf->entity.vsp1;
 	const struct v4l2_mbus_framefmt *source_format;
 	const struct v4l2_mbus_framefmt *sink_format;
-	const struct v4l2_rect *crop = &wpf->crop;
+	const struct v4l2_rect *crop;
 	unsigned int i;
 	u32 srcrpf = 0;
 	u32 outfmt = 0;
@@ -88,6 +88,8 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 				       format->plane_fmt[1].bytesperline);
 	}
 
+	crop = vsp1_rwpf_get_crop(wpf, wpf->entity.config);
+
 	vsp1_wpf_write(wpf, VI6_WPF_HSZCLIP, VI6_WPF_SZCLIP_EN |
 		       (crop->left << VI6_WPF_SZCLIP_OFST_SHIFT) |
 		       (crop->width << VI6_WPF_SZCLIP_SIZE_SHIFT));
-- 
2.7.3

[PATCH v2 32/54] v4l: vsp1: Create a new configure operation to setup modules

[Laurent Pinchart]

The subdev s_stream operation is abused as a generic way to setup
modules at every frame. Move the code out to a new VSP1 entity configure
operation.

Most modules now have an empty s_stream operation that can be removed.
The only exception is the WPF module that needs to perform hardware
configuration when stopping the stream. The code can be simplified
accordingly as we know that that operation never fails.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 231 ++++++++++++++----------------
 drivers/media/platform/vsp1/vsp1_drm.c    |  14 +-
 drivers/media/platform/vsp1/vsp1_entity.h |   3 +
 drivers/media/platform/vsp1/vsp1_hsit.c   |  50 +++----
 drivers/media/platform/vsp1/vsp1_lif.c    |  77 +++++-----
 drivers/media/platform/vsp1/vsp1_lut.c    |  41 +++---
 drivers/media/platform/vsp1/vsp1_pipe.c   |   4 +-
 drivers/media/platform/vsp1/vsp1_rpf.c    |  83 +++++------
 drivers/media/platform/vsp1/vsp1_sru.c    |  91 ++++++------
 drivers/media/platform/vsp1/vsp1_uds.c    | 117 ++++++++-------
 drivers/media/platform/vsp1/vsp1_video.c  |  15 +-
 drivers/media/platform/vsp1/vsp1_wpf.c    | 168 +++++++++++-----------
 12 files changed, 418 insertions(+), 476 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index d3b63056450a..df57ab6d7e3e 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -56,117 +56,7 @@ static const struct v4l2_ctrl_ops bru_ctrl_ops = {
 };
 
 /* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int bru_s_stream(struct v4l2_subdev *subdev, int enable)
-{
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
-	struct vsp1_bru *bru = to_bru(subdev);
-	struct v4l2_mbus_framefmt *format;
-	unsigned int flags;
-	unsigned int i;
-
-	if (!enable)
-		return 0;
-
-	format = vsp1_entity_get_pad_format(&bru->entity, bru->entity.config,
-					    bru->entity.source_pad);
-
-	/* The hardware is extremely flexible but we have no userspace API to
-	 * expose all the parameters, nor is it clear whether we would have use
-	 * cases for all the supported modes. Let's just harcode the parameters
-	 * to sane default values for now.
-	 */
-
-	/* Disable dithering and enable color data normalization unless the
-	 * format at the pipeline output is premultiplied.
-	 */
-	flags = pipe->output ? pipe->output->format.flags : 0;
-	vsp1_bru_write(bru, VI6_BRU_INCTRL,
-		       flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
-		       0 : VI6_BRU_INCTRL_NRM);
-
-	/* Set the background position to cover the whole output image and
-	 * configure its color.
-	 */
-	vsp1_bru_write(bru, VI6_BRU_VIRRPF_SIZE,
-		       (format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
-		       (format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
-	vsp1_bru_write(bru, VI6_BRU_VIRRPF_LOC, 0);
-
-	vsp1_bru_write(bru, VI6_BRU_VIRRPF_COL, bru->bgcolor |
-		       (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
-
-	/* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
-	 * unit with a NOP operation to make BRU input 1 available as the
-	 * Blend/ROP unit B SRC input.
-	 */
-	vsp1_bru_write(bru, VI6_BRU_ROP, VI6_BRU_ROP_DSTSEL_BRUIN(1) |
-		       VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
-		       VI6_BRU_ROP_AROP(VI6_ROP_NOP));
-
-	for (i = 0; i < bru->entity.source_pad; ++i) {
-		bool premultiplied = false;
-		u32 ctrl = 0;
-
-		/* Configure all Blend/ROP units corresponding to an enabled BRU
-		 * input for alpha blending. Blend/ROP units corresponding to
-		 * disabled BRU inputs are used in ROP NOP mode to ignore the
-		 * SRC input.
-		 */
-		if (bru->inputs[i].rpf) {
-			ctrl |= VI6_BRU_CTRL_RBC;
-
-			premultiplied = bru->inputs[i].rpf->format.flags
-				      & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA;
-		} else {
-			ctrl |= VI6_BRU_CTRL_CROP(VI6_ROP_NOP)
-			     |  VI6_BRU_CTRL_AROP(VI6_ROP_NOP);
-		}
-
-		/* Select the virtual RPF as the Blend/ROP unit A DST input to
-		 * serve as a background color.
-		 */
-		if (i == 0)
-			ctrl |= VI6_BRU_CTRL_DSTSEL_VRPF;
-
-		/* Route BRU inputs 0 to 3 as SRC inputs to Blend/ROP units A to
-		 * D in that order. The Blend/ROP unit B SRC is hardwired to the
-		 * ROP unit output, the corresponding register bits must be set
-		 * to 0.
-		 */
-		if (i != 1)
-			ctrl |= VI6_BRU_CTRL_SRCSEL_BRUIN(i);
-
-		vsp1_bru_write(bru, VI6_BRU_CTRL(i), ctrl);
-
-		/* Harcode the blending formula to
-		 *
-		 *	DSTc = DSTc * (1 - SRCa) + SRCc * SRCa
-		 *	DSTa = DSTa * (1 - SRCa) + SRCa
-		 *
-		 * when the SRC input isn't premultiplied, and to
-		 *
-		 *	DSTc = DSTc * (1 - SRCa) + SRCc
-		 *	DSTa = DSTa * (1 - SRCa) + SRCa
-		 *
-		 * otherwise.
-		 */
-		vsp1_bru_write(bru, VI6_BRU_BLD(i),
-			       VI6_BRU_BLD_CCMDX_255_SRC_A |
-			       (premultiplied ? VI6_BRU_BLD_CCMDY_COEFY :
-						VI6_BRU_BLD_CCMDY_SRC_A) |
-			       VI6_BRU_BLD_ACMDX_255_SRC_A |
-			       VI6_BRU_BLD_ACMDY_COEFY |
-			       (0xff << VI6_BRU_BLD_COEFY_SHIFT));
-	}
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
+ * V4L2 Subdevice Operations
  */
 
 /*
@@ -395,14 +285,6 @@ static int bru_set_selection(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops bru_video_ops = {
-	.s_stream = bru_s_stream,
-};
-
 static struct v4l2_subdev_pad_ops bru_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = bru_enum_mbus_code,
@@ -414,11 +296,119 @@ static struct v4l2_subdev_pad_ops bru_pad_ops = {
 };
 
 static struct v4l2_subdev_ops bru_ops = {
-	.video	= &bru_video_ops,
 	.pad    = &bru_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void bru_configure(struct vsp1_entity *entity)
+{
+	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
+	struct vsp1_bru *bru = to_bru(&entity->subdev);
+	struct v4l2_mbus_framefmt *format;
+	unsigned int flags;
+	unsigned int i;
+
+	format = vsp1_entity_get_pad_format(&bru->entity, bru->entity.config,
+					    bru->entity.source_pad);
+
+	/* The hardware is extremely flexible but we have no userspace API to
+	 * expose all the parameters, nor is it clear whether we would have use
+	 * cases for all the supported modes. Let's just harcode the parameters
+	 * to sane default values for now.
+	 */
+
+	/* Disable dithering and enable color data normalization unless the
+	 * format at the pipeline output is premultiplied.
+	 */
+	flags = pipe->output ? pipe->output->format.flags : 0;
+	vsp1_bru_write(bru, VI6_BRU_INCTRL,
+		       flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
+		       0 : VI6_BRU_INCTRL_NRM);
+
+	/* Set the background position to cover the whole output image and
+	 * configure its color.
+	 */
+	vsp1_bru_write(bru, VI6_BRU_VIRRPF_SIZE,
+		       (format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
+		       (format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
+	vsp1_bru_write(bru, VI6_BRU_VIRRPF_LOC, 0);
+
+	vsp1_bru_write(bru, VI6_BRU_VIRRPF_COL, bru->bgcolor |
+		       (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
+
+	/* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
+	 * unit with a NOP operation to make BRU input 1 available as the
+	 * Blend/ROP unit B SRC input.
+	 */
+	vsp1_bru_write(bru, VI6_BRU_ROP, VI6_BRU_ROP_DSTSEL_BRUIN(1) |
+		       VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
+		       VI6_BRU_ROP_AROP(VI6_ROP_NOP));
+
+	for (i = 0; i < bru->entity.source_pad; ++i) {
+		bool premultiplied = false;
+		u32 ctrl = 0;
+
+		/* Configure all Blend/ROP units corresponding to an enabled BRU
+		 * input for alpha blending. Blend/ROP units corresponding to
+		 * disabled BRU inputs are used in ROP NOP mode to ignore the
+		 * SRC input.
+		 */
+		if (bru->inputs[i].rpf) {
+			ctrl |= VI6_BRU_CTRL_RBC;
+
+			premultiplied = bru->inputs[i].rpf->format.flags
+				      & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA;
+		} else {
+			ctrl |= VI6_BRU_CTRL_CROP(VI6_ROP_NOP)
+			     |  VI6_BRU_CTRL_AROP(VI6_ROP_NOP);
+		}
+
+		/* Select the virtual RPF as the Blend/ROP unit A DST input to
+		 * serve as a background color.
+		 */
+		if (i == 0)
+			ctrl |= VI6_BRU_CTRL_DSTSEL_VRPF;
+
+		/* Route BRU inputs 0 to 3 as SRC inputs to Blend/ROP units A to
+		 * D in that order. The Blend/ROP unit B SRC is hardwired to the
+		 * ROP unit output, the corresponding register bits must be set
+		 * to 0.
+		 */
+		if (i != 1)
+			ctrl |= VI6_BRU_CTRL_SRCSEL_BRUIN(i);
+
+		vsp1_bru_write(bru, VI6_BRU_CTRL(i), ctrl);
+
+		/* Harcode the blending formula to
+		 *
+		 *	DSTc = DSTc * (1 - SRCa) + SRCc * SRCa
+		 *	DSTa = DSTa * (1 - SRCa) + SRCa
+		 *
+		 * when the SRC input isn't premultiplied, and to
+		 *
+		 *	DSTc = DSTc * (1 - SRCa) + SRCc
+		 *	DSTa = DSTa * (1 - SRCa) + SRCa
+		 *
+		 * otherwise.
+		 */
+		vsp1_bru_write(bru, VI6_BRU_BLD(i),
+			       VI6_BRU_BLD_CCMDX_255_SRC_A |
+			       (premultiplied ? VI6_BRU_BLD_CCMDY_COEFY :
+						VI6_BRU_BLD_CCMDY_SRC_A) |
+			       VI6_BRU_BLD_ACMDX_255_SRC_A |
+			       VI6_BRU_BLD_ACMDY_COEFY |
+			       (0xff << VI6_BRU_BLD_COEFY_SHIFT));
+	}
+}
+
+static const struct vsp1_entity_operations bru_entity_ops = {
+	.configure = bru_configure,
+};
+
+/* -----------------------------------------------------------------------------
  * Initialization and Cleanup
  */
 
@@ -431,6 +421,7 @@ struct vsp1_bru *vsp1_bru_create(struct vsp1_device *vsp1)
 	if (bru == NULL)
 		return ERR_PTR(-ENOMEM);
 
+	bru->entity.ops = &bru_entity_ops;
 	bru->entity.type = VSP1_ENTITY_BRU;
 
 	ret = vsp1_entity_init(vsp1, &bru->entity, "bru",
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index acbf36d315b9..bec7a651d152 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -448,7 +448,6 @@ void vsp1_du_atomic_flush(struct device *dev)
 	struct vsp1_entity *entity;
 	unsigned long flags;
 	bool stop = false;
-	int ret;
 
 	list_for_each_entry(entity, &pipe->entities, list_pipe) {
 		/* Disconnect unused RPFs from the pipeline. */
@@ -464,19 +463,16 @@ void vsp1_du_atomic_flush(struct device *dev)
 
 		vsp1_entity_route_setup(entity);
 
-		ret = v4l2_subdev_call(&entity->subdev, video,
-				       s_stream, 1);
-		if (ret < 0) {
-			dev_err(vsp1->dev,
-				"DRM pipeline start failure on entity %s\n",
-				entity->subdev.name);
-			return;
-		}
+		if (entity->ops->configure)
+			entity->ops->configure(entity);
 
 		if (entity->type == VSP1_ENTITY_RPF)
 			vsp1_rwpf_set_memory(to_rwpf(&entity->subdev));
 	}
 
+	/* We know that the WPF s_stream operation never fails. */
+	v4l2_subdev_call(&pipe->output->entity.subdev, video, s_stream, 1);
+
 	vsp1_dl_list_commit(pipe->dl);
 	pipe->dl = NULL;
 
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 1935ae9289ba..3ce74c6f12e1 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -59,10 +59,13 @@ struct vsp1_route {
  * @set_memory:	Setup memory buffer access. This operation applies the settings
  *		stored in the rwpf mem field to the hardware. Valid for RPF and
  *		WPF only.
+ * @configure:	Setup the hardware based on the entity state (pipeline, formats,
+ *		selection rectangles, ...)
  */
 struct vsp1_entity_operations {
 	void (*destroy)(struct vsp1_entity *);
 	void (*set_memory)(struct vsp1_entity *);
+	void (*configure)(struct vsp1_entity *);
 };
 
 struct vsp1_entity {
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index 6565ef62c376..b935a62e6399 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -32,26 +32,7 @@ static inline void vsp1_hsit_write(struct vsp1_hsit *hsit, u32 reg, u32 data)
 }
 
 /* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int hsit_s_stream(struct v4l2_subdev *subdev, int enable)
-{
-	struct vsp1_hsit *hsit = to_hsit(subdev);
-
-	if (!enable)
-		return 0;
-
-	if (hsit->inverse)
-		vsp1_hsit_write(hsit, VI6_HSI_CTRL, VI6_HSI_CTRL_EN);
-	else
-		vsp1_hsit_write(hsit, VI6_HST_CTRL, VI6_HST_CTRL_EN);
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
+ * V4L2 Subdevice Operations
  */
 
 static int hsit_enum_mbus_code(struct v4l2_subdev *subdev,
@@ -167,14 +148,6 @@ static int hsit_set_format(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops hsit_video_ops = {
-	.s_stream = hsit_s_stream,
-};
-
 static struct v4l2_subdev_pad_ops hsit_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = hsit_enum_mbus_code,
@@ -184,11 +157,28 @@ static struct v4l2_subdev_pad_ops hsit_pad_ops = {
 };
 
 static struct v4l2_subdev_ops hsit_ops = {
-	.video	= &hsit_video_ops,
 	.pad    = &hsit_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void hsit_configure(struct vsp1_entity *entity)
+{
+	struct vsp1_hsit *hsit = to_hsit(&entity->subdev);
+
+	if (hsit->inverse)
+		vsp1_hsit_write(hsit, VI6_HSI_CTRL, VI6_HSI_CTRL_EN);
+	else
+		vsp1_hsit_write(hsit, VI6_HST_CTRL, VI6_HST_CTRL_EN);
+}
+
+static const struct vsp1_entity_operations hsit_entity_ops = {
+	.configure = hsit_configure,
+};
+
+/* -----------------------------------------------------------------------------
  * Initialization and Cleanup
  */
 
@@ -203,6 +193,8 @@ struct vsp1_hsit *vsp1_hsit_create(struct vsp1_device *vsp1, bool inverse)
 
 	hsit->inverse = inverse;
 
+	hsit->entity.ops = &hsit_entity_ops;
+
 	if (inverse)
 		hsit->entity.type = VSP1_ENTITY_HSI;
 	else
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 9ab40d843578..d3b94055f23c 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -32,41 +32,7 @@ static inline void vsp1_lif_write(struct vsp1_lif *lif, u32 reg, u32 data)
 }
 
 /* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int lif_s_stream(struct v4l2_subdev *subdev, int enable)
-{
-	const struct v4l2_mbus_framefmt *format;
-	struct vsp1_lif *lif = to_lif(subdev);
-	unsigned int hbth = 1300;
-	unsigned int obth = 400;
-	unsigned int lbth = 200;
-
-	if (!enable) {
-		vsp1_write(lif->entity.vsp1, VI6_LIF_CTRL, 0);
-		return 0;
-	}
-
-	format = vsp1_entity_get_pad_format(&lif->entity, lif->entity.config,
-					    LIF_PAD_SOURCE);
-
-	obth = min(obth, (format->width + 1) / 2 * format->height - 4);
-
-	vsp1_lif_write(lif, VI6_LIF_CSBTH,
-			(hbth << VI6_LIF_CSBTH_HBTH_SHIFT) |
-			(lbth << VI6_LIF_CSBTH_LBTH_SHIFT));
-
-	vsp1_lif_write(lif, VI6_LIF_CTRL,
-			(obth << VI6_LIF_CTRL_OBTH_SHIFT) |
-			(format->code == 0 ? VI6_LIF_CTRL_CFMT : 0) |
-			VI6_LIF_CTRL_REQSEL | VI6_LIF_CTRL_LIF_EN);
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
+ * V4L2 Subdevice Operations
  */
 
 static int lif_enum_mbus_code(struct v4l2_subdev *subdev,
@@ -201,14 +167,6 @@ static int lif_set_format(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops lif_video_ops = {
-	.s_stream = lif_s_stream,
-};
-
 static struct v4l2_subdev_pad_ops lif_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = lif_enum_mbus_code,
@@ -218,11 +176,41 @@ static struct v4l2_subdev_pad_ops lif_pad_ops = {
 };
 
 static struct v4l2_subdev_ops lif_ops = {
-	.video	= &lif_video_ops,
 	.pad    = &lif_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void lif_configure(struct vsp1_entity *entity)
+{
+	const struct v4l2_mbus_framefmt *format;
+	struct vsp1_lif *lif = to_lif(&entity->subdev);
+	unsigned int hbth = 1300;
+	unsigned int obth = 400;
+	unsigned int lbth = 200;
+
+	format = vsp1_entity_get_pad_format(&lif->entity, lif->entity.config,
+					    LIF_PAD_SOURCE);
+
+	obth = min(obth, (format->width + 1) / 2 * format->height - 4);
+
+	vsp1_lif_write(lif, VI6_LIF_CSBTH,
+			(hbth << VI6_LIF_CSBTH_HBTH_SHIFT) |
+			(lbth << VI6_LIF_CSBTH_LBTH_SHIFT));
+
+	vsp1_lif_write(lif, VI6_LIF_CTRL,
+			(obth << VI6_LIF_CTRL_OBTH_SHIFT) |
+			(format->code == 0 ? VI6_LIF_CTRL_CFMT : 0) |
+			VI6_LIF_CTRL_REQSEL | VI6_LIF_CTRL_LIF_EN);
+}
+
+static const struct vsp1_entity_operations lif_entity_ops = {
+	.configure = lif_configure,
+};
+
+/* -----------------------------------------------------------------------------
  * Initialization and Cleanup
  */
 
@@ -235,6 +223,7 @@ struct vsp1_lif *vsp1_lif_create(struct vsp1_device *vsp1)
 	if (lif == NULL)
 		return ERR_PTR(-ENOMEM);
 
+	lif->entity.ops = &lif_entity_ops;
 	lif->entity.type = VSP1_ENTITY_LIF;
 
 	ret = vsp1_entity_init(vsp1, &lif->entity, "lif", 2, &lif_ops,
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 9510424113b2..1e8d43460d49 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -36,7 +36,7 @@ static inline void vsp1_lut_write(struct vsp1_lut *lut, u32 reg, u32 data)
  * V4L2 Subdevice Core Operations
  */
 
-static void lut_configure(struct vsp1_lut *lut, struct vsp1_lut_config *config)
+static void lut_set_table(struct vsp1_lut *lut, struct vsp1_lut_config *config)
 {
 	memcpy_toio(lut->entity.vsp1->mmio + VI6_LUT_TABLE, config->lut,
 		    sizeof(config->lut));
@@ -48,7 +48,7 @@ static long lut_ioctl(struct v4l2_subdev *subdev, unsigned int cmd, void *arg)
 
 	switch (cmd) {
 	case VIDIOC_VSP1_LUT_CONFIG:
-		lut_configure(lut, arg);
+		lut_set_table(lut, arg);
 		return 0;
 
 	default:
@@ -57,22 +57,6 @@ static long lut_ioctl(struct v4l2_subdev *subdev, unsigned int cmd, void *arg)
 }
 
 /* -----------------------------------------------------------------------------
- * V4L2 Subdevice Video Operations
- */
-
-static int lut_s_stream(struct v4l2_subdev *subdev, int enable)
-{
-	struct vsp1_lut *lut = to_lut(subdev);
-
-	if (!enable)
-		return 0;
-
-	vsp1_lut_write(lut, VI6_LUT_CTRL, VI6_LUT_CTRL_EN);
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
  * V4L2 Subdevice Pad Operations
  */
 
@@ -218,10 +202,6 @@ static struct v4l2_subdev_core_ops lut_core_ops = {
 	.ioctl = lut_ioctl,
 };
 
-static struct v4l2_subdev_video_ops lut_video_ops = {
-	.s_stream = lut_s_stream,
-};
-
 static struct v4l2_subdev_pad_ops lut_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = lut_enum_mbus_code,
@@ -232,11 +212,25 @@ static struct v4l2_subdev_pad_ops lut_pad_ops = {
 
 static struct v4l2_subdev_ops lut_ops = {
 	.core	= &lut_core_ops,
-	.video	= &lut_video_ops,
 	.pad    = &lut_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void lut_configure(struct vsp1_entity *entity)
+{
+	struct vsp1_lut *lut = to_lut(&entity->subdev);
+
+	vsp1_lut_write(lut, VI6_LUT_CTRL, VI6_LUT_CTRL_EN);
+}
+
+static const struct vsp1_entity_operations lut_entity_ops = {
+	.configure = lut_configure,
+};
+
+/* -----------------------------------------------------------------------------
  * Initialization and Cleanup
  */
 
@@ -249,6 +243,7 @@ struct vsp1_lut *vsp1_lut_create(struct vsp1_device *vsp1)
 	if (lut == NULL)
 		return ERR_PTR(-ENOMEM);
 
+	lut->entity.ops = &lut_entity_ops;
 	lut->entity.type = VSP1_ENTITY_LUT;
 
 	ret = vsp1_entity_init(vsp1, &lut->entity, "lut", 2, &lut_ops,
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 3311db18f40b..fe2538d5bed1 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -253,10 +253,10 @@ int vsp1_pipeline_stop(struct vsp1_pipeline *pipe)
 		if (entity->route && entity->route->reg)
 			vsp1_write(entity->vsp1, entity->route->reg,
 				   VI6_DPR_NODE_UNUSED);
-
-		v4l2_subdev_call(&entity->subdev, video, s_stream, 0);
 	}
 
+	v4l2_subdev_call(&pipe->output->entity.subdev, video, s_stream, 0);
+
 	return ret;
 }
 
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 198487952418..acc88b4a449b 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -33,13 +33,43 @@ static inline void vsp1_rpf_write(struct vsp1_rwpf *rpf, u32 reg, u32 data)
 }
 
 /* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
+ * V4L2 Subdevice Operations
  */
 
-static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
+static struct v4l2_subdev_pad_ops rpf_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
+	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
+	.enum_frame_size = vsp1_rwpf_enum_frame_size,
+	.get_fmt = vsp1_rwpf_get_format,
+	.set_fmt = vsp1_rwpf_set_format,
+	.get_selection = vsp1_rwpf_get_selection,
+	.set_selection = vsp1_rwpf_set_selection,
+};
+
+static struct v4l2_subdev_ops rpf_ops = {
+	.pad    = &rpf_pad_ops,
+};
+
+/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void rpf_set_memory(struct vsp1_entity *entity)
 {
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
-	struct vsp1_rwpf *rpf = to_rwpf(subdev);
+	struct vsp1_rwpf *rpf = entity_to_rwpf(entity);
+
+	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
+		       rpf->mem.addr[0] + rpf->offsets[0]);
+	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
+		       rpf->mem.addr[1] + rpf->offsets[1]);
+	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
+		       rpf->mem.addr[2] + rpf->offsets[1]);
+}
+
+static void rpf_configure(struct vsp1_entity *entity)
+{
+	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
+	struct vsp1_rwpf *rpf = to_rwpf(&entity->subdev);
 	const struct vsp1_format_info *fmtinfo = rpf->fmtinfo;
 	const struct v4l2_pix_format_mplane *format = &rpf->format;
 	const struct v4l2_mbus_framefmt *source_format;
@@ -50,9 +80,6 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 	u32 pstride;
 	u32 infmt;
 
-	if (!enable)
-		return 0;
-
 	/* Source size, stride and crop offsets.
 	 *
 	 * The crop offsets correspond to the location of the crop rectangle top
@@ -136,51 +163,11 @@ static int rpf_s_stream(struct v4l2_subdev *subdev, int enable)
 
 	vsp1_rpf_write(rpf, VI6_RPF_MSK_CTRL, 0);
 	vsp1_rpf_write(rpf, VI6_RPF_CKEY_CTRL, 0);
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops rpf_video_ops = {
-	.s_stream = rpf_s_stream,
-};
-
-static struct v4l2_subdev_pad_ops rpf_pad_ops = {
-	.init_cfg = vsp1_entity_init_cfg,
-	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
-	.enum_frame_size = vsp1_rwpf_enum_frame_size,
-	.get_fmt = vsp1_rwpf_get_format,
-	.set_fmt = vsp1_rwpf_set_format,
-	.get_selection = vsp1_rwpf_get_selection,
-	.set_selection = vsp1_rwpf_set_selection,
-};
-
-static struct v4l2_subdev_ops rpf_ops = {
-	.video	= &rpf_video_ops,
-	.pad    = &rpf_pad_ops,
-};
-
-/* -----------------------------------------------------------------------------
- * VSP1 Entity Operations
- */
-
-static void rpf_set_memory(struct vsp1_entity *entity)
-{
-	struct vsp1_rwpf *rpf = entity_to_rwpf(entity);
-
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
-		       rpf->mem.addr[0] + rpf->offsets[0]);
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
-		       rpf->mem.addr[1] + rpf->offsets[1]);
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
-		       rpf->mem.addr[2] + rpf->offsets[1]);
 }
 
 static const struct vsp1_entity_operations rpf_entity_ops = {
 	.set_memory = rpf_set_memory,
+	.configure = rpf_configure,
 };
 
 /* -----------------------------------------------------------------------------
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index a67fa0c9fe3b..82a8ee202f1a 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -103,47 +103,7 @@ static const struct v4l2_ctrl_config sru_intensity_control = {
 };
 
 /* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int sru_s_stream(struct v4l2_subdev *subdev, int enable)
-{
-	const struct vsp1_sru_param *param;
-	struct vsp1_sru *sru = to_sru(subdev);
-	struct v4l2_mbus_framefmt *input;
-	struct v4l2_mbus_framefmt *output;
-	u32 ctrl0;
-
-	if (!enable)
-		return 0;
-
-	input = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
-					   SRU_PAD_SINK);
-	output = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
-					    SRU_PAD_SOURCE);
-
-	if (input->code == MEDIA_BUS_FMT_ARGB8888_1X32)
-		ctrl0 = VI6_SRU_CTRL0_PARAM2 | VI6_SRU_CTRL0_PARAM3
-		      | VI6_SRU_CTRL0_PARAM4;
-	else
-		ctrl0 = VI6_SRU_CTRL0_PARAM3;
-
-	if (input->width != output->width)
-		ctrl0 |= VI6_SRU_CTRL0_MODE_UPSCALE;
-
-	param = &vsp1_sru_params[sru->intensity - 1];
-
-	ctrl0 |= param->ctrl0;
-
-	vsp1_sru_write(sru, VI6_SRU_CTRL0, ctrl0);
-	vsp1_sru_write(sru, VI6_SRU_CTRL1, VI6_SRU_CTRL1_PARAM5);
-	vsp1_sru_write(sru, VI6_SRU_CTRL2, param->ctrl2);
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Pad Operations
+ * V4L2 Subdevice Operations
  */
 
 static int sru_enum_mbus_code(struct v4l2_subdev *subdev,
@@ -319,14 +279,6 @@ static int sru_set_format(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops sru_video_ops = {
-	.s_stream = sru_s_stream,
-};
-
 static struct v4l2_subdev_pad_ops sru_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = sru_enum_mbus_code,
@@ -336,11 +288,49 @@ static struct v4l2_subdev_pad_ops sru_pad_ops = {
 };
 
 static struct v4l2_subdev_ops sru_ops = {
-	.video	= &sru_video_ops,
 	.pad    = &sru_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void sru_configure(struct vsp1_entity *entity)
+{
+	const struct vsp1_sru_param *param;
+	struct vsp1_sru *sru = to_sru(&entity->subdev);
+	struct v4l2_mbus_framefmt *input;
+	struct v4l2_mbus_framefmt *output;
+	u32 ctrl0;
+
+	input = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
+					   SRU_PAD_SINK);
+	output = vsp1_entity_get_pad_format(&sru->entity, sru->entity.config,
+					    SRU_PAD_SOURCE);
+
+	if (input->code == MEDIA_BUS_FMT_ARGB8888_1X32)
+		ctrl0 = VI6_SRU_CTRL0_PARAM2 | VI6_SRU_CTRL0_PARAM3
+		      | VI6_SRU_CTRL0_PARAM4;
+	else
+		ctrl0 = VI6_SRU_CTRL0_PARAM3;
+
+	if (input->width != output->width)
+		ctrl0 |= VI6_SRU_CTRL0_MODE_UPSCALE;
+
+	param = &vsp1_sru_params[sru->intensity - 1];
+
+	ctrl0 |= param->ctrl0;
+
+	vsp1_sru_write(sru, VI6_SRU_CTRL0, ctrl0);
+	vsp1_sru_write(sru, VI6_SRU_CTRL1, VI6_SRU_CTRL1_PARAM5);
+	vsp1_sru_write(sru, VI6_SRU_CTRL2, param->ctrl2);
+}
+
+static const struct vsp1_entity_operations sru_entity_ops = {
+	.configure = sru_configure,
+};
+
+/* -----------------------------------------------------------------------------
  * Initialization and Cleanup
  */
 
@@ -353,6 +343,7 @@ struct vsp1_sru *vsp1_sru_create(struct vsp1_device *vsp1)
 	if (sru == NULL)
 		return ERR_PTR(-ENOMEM);
 
+	sru->entity.ops = &sru_entity_ops;
 	sru->entity.type = VSP1_ENTITY_SRU;
 
 	ret = vsp1_entity_init(vsp1, &sru->entity, "sru", 2, &sru_ops,
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index c2209823140e..88c44f450f88 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -105,62 +105,6 @@ static unsigned int uds_compute_ratio(unsigned int input, unsigned int output)
 }
 
 /* -----------------------------------------------------------------------------
- * V4L2 Subdevice Core Operations
- */
-
-static int uds_s_stream(struct v4l2_subdev *subdev, int enable)
-{
-	struct vsp1_uds *uds = to_uds(subdev);
-	const struct v4l2_mbus_framefmt *output;
-	const struct v4l2_mbus_framefmt *input;
-	unsigned int hscale;
-	unsigned int vscale;
-	bool multitap;
-
-	if (!enable)
-		return 0;
-
-	input = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
-					   UDS_PAD_SINK);
-	output = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
-					    UDS_PAD_SOURCE);
-
-	hscale = uds_compute_ratio(input->width, output->width);
-	vscale = uds_compute_ratio(input->height, output->height);
-
-	dev_dbg(uds->entity.vsp1->dev, "hscale %u vscale %u\n", hscale, vscale);
-
-	/* Multi-tap scaling can't be enabled along with alpha scaling when
-	 * scaling down with a factor lower than or equal to 1/2 in either
-	 * direction.
-	 */
-	if (uds->scale_alpha && (hscale >= 8192 || vscale >= 8192))
-		multitap = false;
-	else
-		multitap = true;
-
-	vsp1_uds_write(uds, VI6_UDS_CTRL,
-		       (uds->scale_alpha ? VI6_UDS_CTRL_AON : 0) |
-		       (multitap ? VI6_UDS_CTRL_BC : 0));
-
-	vsp1_uds_write(uds, VI6_UDS_PASS_BWIDTH,
-		       (uds_passband_width(hscale)
-				<< VI6_UDS_PASS_BWIDTH_H_SHIFT) |
-		       (uds_passband_width(vscale)
-				<< VI6_UDS_PASS_BWIDTH_V_SHIFT));
-
-	/* Set the scaling ratios and the output size. */
-	vsp1_uds_write(uds, VI6_UDS_SCALE,
-		       (hscale << VI6_UDS_SCALE_HFRAC_SHIFT) |
-		       (vscale << VI6_UDS_SCALE_VFRAC_SHIFT));
-	vsp1_uds_write(uds, VI6_UDS_CLIP_SIZE,
-		       (output->width << VI6_UDS_CLIP_SIZE_HSIZE_SHIFT) |
-		       (output->height << VI6_UDS_CLIP_SIZE_VSIZE_SHIFT));
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
  * V4L2 Subdevice Pad Operations
  */
 
@@ -321,10 +265,6 @@ static int uds_set_format(struct v4l2_subdev *subdev,
  * V4L2 Subdevice Operations
  */
 
-static struct v4l2_subdev_video_ops uds_video_ops = {
-	.s_stream = uds_s_stream,
-};
-
 static struct v4l2_subdev_pad_ops uds_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = uds_enum_mbus_code,
@@ -334,11 +274,65 @@ static struct v4l2_subdev_pad_ops uds_pad_ops = {
 };
 
 static struct v4l2_subdev_ops uds_ops = {
-	.video	= &uds_video_ops,
 	.pad    = &uds_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void uds_configure(struct vsp1_entity *entity)
+{
+	struct vsp1_uds *uds = to_uds(&entity->subdev);
+	const struct v4l2_mbus_framefmt *output;
+	const struct v4l2_mbus_framefmt *input;
+	unsigned int hscale;
+	unsigned int vscale;
+	bool multitap;
+
+	input = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
+					   UDS_PAD_SINK);
+	output = vsp1_entity_get_pad_format(&uds->entity, uds->entity.config,
+					    UDS_PAD_SOURCE);
+
+	hscale = uds_compute_ratio(input->width, output->width);
+	vscale = uds_compute_ratio(input->height, output->height);
+
+	dev_dbg(uds->entity.vsp1->dev, "hscale %u vscale %u\n", hscale, vscale);
+
+	/* Multi-tap scaling can't be enabled along with alpha scaling when
+	 * scaling down with a factor lower than or equal to 1/2 in either
+	 * direction.
+	 */
+	if (uds->scale_alpha && (hscale >= 8192 || vscale >= 8192))
+		multitap = false;
+	else
+		multitap = true;
+
+	vsp1_uds_write(uds, VI6_UDS_CTRL,
+		       (uds->scale_alpha ? VI6_UDS_CTRL_AON : 0) |
+		       (multitap ? VI6_UDS_CTRL_BC : 0));
+
+	vsp1_uds_write(uds, VI6_UDS_PASS_BWIDTH,
+		       (uds_passband_width(hscale)
+				<< VI6_UDS_PASS_BWIDTH_H_SHIFT) |
+		       (uds_passband_width(vscale)
+				<< VI6_UDS_PASS_BWIDTH_V_SHIFT));
+
+	/* Set the scaling ratios and the output size. */
+	vsp1_uds_write(uds, VI6_UDS_SCALE,
+		       (hscale << VI6_UDS_SCALE_HFRAC_SHIFT) |
+		       (vscale << VI6_UDS_SCALE_VFRAC_SHIFT));
+	vsp1_uds_write(uds, VI6_UDS_CLIP_SIZE,
+		       (output->width << VI6_UDS_CLIP_SIZE_HSIZE_SHIFT) |
+		       (output->height << VI6_UDS_CLIP_SIZE_VSIZE_SHIFT));
+}
+
+static const struct vsp1_entity_operations uds_entity_ops = {
+	.configure = uds_configure,
+};
+
+/* -----------------------------------------------------------------------------
  * Initialization and Cleanup
  */
 
@@ -352,6 +346,7 @@ struct vsp1_uds *vsp1_uds_create(struct vsp1_device *vsp1, unsigned int index)
 	if (uds == NULL)
 		return ERR_PTR(-ENOMEM);
 
+	uds->entity.ops = &uds_entity_ops;
 	uds->entity.type = VSP1_ENTITY_UDS;
 	uds->entity.index = index;
 
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index d4a092c8ece3..a3f1145c8a79 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -591,7 +591,6 @@ static void vsp1_video_buffer_queue(struct vb2_buffer *vb)
 static int vsp1_video_setup_pipeline(struct vsp1_pipeline *pipe)
 {
 	struct vsp1_entity *entity;
-	int ret;
 
 	/* Prepare the display list. */
 	pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
@@ -619,18 +618,14 @@ static int vsp1_video_setup_pipeline(struct vsp1_pipeline *pipe)
 	list_for_each_entry(entity, &pipe->entities, list_pipe) {
 		vsp1_entity_route_setup(entity);
 
-		ret = v4l2_subdev_call(&entity->subdev, video, s_stream, 1);
-		if (ret < 0)
-			goto error;
+		if (entity->ops->configure)
+			entity->ops->configure(entity);
 	}
 
-	return 0;
+	/* We know that the WPF s_stream operation never fails. */
+	v4l2_subdev_call(&pipe->output->entity.subdev, video, s_stream, 1);
 
-error:
-	vsp1_dl_list_put(pipe->dl);
-	pipe->dl = NULL;
-
-	return ret;
+	return 0;
 }
 
 static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 962b43b7f99e..65481930b218 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -39,55 +39,78 @@ static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf, u32 reg, u32 data)
 
 static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 {
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&subdev->entity);
 	struct vsp1_rwpf *wpf = to_rwpf(subdev);
 	struct vsp1_device *vsp1 = wpf->entity.vsp1;
-	const struct v4l2_mbus_framefmt *source_format;
-	const struct v4l2_mbus_framefmt *sink_format;
-	const struct v4l2_rect *crop;
-	unsigned int i;
-	u32 srcrpf = 0;
-	u32 outfmt = 0;
 
-	if (!enable) {
-		vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index), 0);
-		vsp1_write(vsp1, wpf->entity.index * VI6_WPF_OFFSET +
-			   VI6_WPF_SRCRPF, 0);
+	if (enable)
 		return 0;
-	}
 
-	/* Sources. If the pipeline has a single input and BRU is not used,
-	 * configure it as the master layer. Otherwise configure all
-	 * inputs as sub-layers and select the virtual RPF as the master
-	 * layer.
+	/* Write to registers directly when stopping the stream as there will be
+	 * no pipeline run to apply the display list.
 	 */
-	for (i = 0; i < vsp1->info->rpf_count; ++i) {
-		struct vsp1_rwpf *input = pipe->inputs[i];
+	vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index), 0);
+	vsp1_write(vsp1, wpf->entity.index * VI6_WPF_OFFSET +
+		   VI6_WPF_SRCRPF, 0);
 
-		if (!input)
-			continue;
+	return 0;
+}
 
-		srcrpf |= (!pipe->bru && pipe->num_inputs == 1)
-			? VI6_WPF_SRCRPF_RPF_ACT_MST(input->entity.index)
-			: VI6_WPF_SRCRPF_RPF_ACT_SUB(input->entity.index);
-	}
+/* -----------------------------------------------------------------------------
+ * V4L2 Subdevice Operations
+ */
 
-	if (pipe->bru || pipe->num_inputs > 1)
-		srcrpf |= VI6_WPF_SRCRPF_VIRACT_MST;
+static struct v4l2_subdev_video_ops wpf_video_ops = {
+	.s_stream = wpf_s_stream,
+};
 
-	vsp1_wpf_write(wpf, VI6_WPF_SRCRPF, srcrpf);
+static struct v4l2_subdev_pad_ops wpf_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
+	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
+	.enum_frame_size = vsp1_rwpf_enum_frame_size,
+	.get_fmt = vsp1_rwpf_get_format,
+	.set_fmt = vsp1_rwpf_set_format,
+	.get_selection = vsp1_rwpf_get_selection,
+	.set_selection = vsp1_rwpf_set_selection,
+};
 
-	/* Destination stride. */
-	if (!pipe->lif) {
-		struct v4l2_pix_format_mplane *format = &wpf->format;
+static struct v4l2_subdev_ops wpf_ops = {
+	.video	= &wpf_video_ops,
+	.pad    = &wpf_pad_ops,
+};
 
-		vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_Y,
-			       format->plane_fmt[0].bytesperline);
-		if (format->num_planes > 1)
-			vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_C,
-				       format->plane_fmt[1].bytesperline);
-	}
+/* -----------------------------------------------------------------------------
+ * VSP1 Entity Operations
+ */
+
+static void vsp1_wpf_destroy(struct vsp1_entity *entity)
+{
+	struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+
+	vsp1_dlm_destroy(wpf->dlm);
+}
+
+static void wpf_set_memory(struct vsp1_entity *entity)
+{
+	struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, wpf->mem.addr[0]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, wpf->mem.addr[1]);
+	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
+}
+
+static void wpf_configure(struct vsp1_entity *entity)
+{
+	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
+	struct vsp1_rwpf *wpf = to_rwpf(&entity->subdev);
+	struct vsp1_device *vsp1 = wpf->entity.vsp1;
+	const struct v4l2_mbus_framefmt *source_format;
+	const struct v4l2_mbus_framefmt *sink_format;
+	const struct v4l2_rect *crop;
+	unsigned int i;
+	u32 outfmt = 0;
+	u32 srcrpf = 0;
 
+	/* Cropping */
 	crop = vsp1_rwpf_get_crop(wpf, wpf->entity.config);
 
 	vsp1_wpf_write(wpf, VI6_WPF_HSZCLIP, VI6_WPF_SZCLIP_EN |
@@ -106,6 +129,7 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 						   RWPF_PAD_SOURCE);
 
 	if (!pipe->lif) {
+		const struct v4l2_pix_format_mplane *format = &wpf->format;
 		const struct vsp1_format_info *fmtinfo = wpf->fmtinfo;
 
 		outfmt = fmtinfo->hwfmt << VI6_WPF_OUTFMT_WRFMT_SHIFT;
@@ -117,6 +141,13 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 		if (fmtinfo->swap_uv)
 			outfmt |= VI6_WPF_OUTFMT_SPUVS;
 
+		/* Destination stride and byte swapping. */
+		vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_Y,
+			       format->plane_fmt[0].bytesperline);
+		if (format->num_planes > 1)
+			vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_C,
+				       format->plane_fmt[1].bytesperline);
+
 		vsp1_wpf_write(wpf, VI6_WPF_DSWAP, fmtinfo->swap);
 	}
 
@@ -131,60 +162,37 @@ static int wpf_s_stream(struct v4l2_subdev *subdev, int enable)
 
 	vsp1_mod_write(&wpf->entity, VI6_WPF_WRBCK_CTRL, 0);
 
-	/* Enable interrupts */
-	vsp1_write(vsp1, VI6_WPF_IRQ_STA(wpf->entity.index), 0);
-	vsp1_write(vsp1, VI6_WPF_IRQ_ENB(wpf->entity.index),
-		   VI6_WFP_IRQ_ENB_FREE);
-
-	return 0;
-}
-
-/* -----------------------------------------------------------------------------
- * V4L2 Subdevice Operations
- */
-
-static struct v4l2_subdev_video_ops wpf_video_ops = {
-	.s_stream = wpf_s_stream,
-};
-
-static struct v4l2_subdev_pad_ops wpf_pad_ops = {
-	.init_cfg = vsp1_entity_init_cfg,
-	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
-	.enum_frame_size = vsp1_rwpf_enum_frame_size,
-	.get_fmt = vsp1_rwpf_get_format,
-	.set_fmt = vsp1_rwpf_set_format,
-	.get_selection = vsp1_rwpf_get_selection,
-	.set_selection = vsp1_rwpf_set_selection,
-};
-
-static struct v4l2_subdev_ops wpf_ops = {
-	.video	= &wpf_video_ops,
-	.pad    = &wpf_pad_ops,
-};
+	/* Sources. If the pipeline has a single input and BRU is not used,
+	 * configure it as the master layer. Otherwise configure all
+	 * inputs as sub-layers and select the virtual RPF as the master
+	 * layer.
+	 */
+	for (i = 0; i < vsp1->info->rpf_count; ++i) {
+		struct vsp1_rwpf *input = pipe->inputs[i];
 
-/* -----------------------------------------------------------------------------
- * VSP1 Entity Operations
- */
+		if (!input)
+			continue;
 
-static void vsp1_wpf_destroy(struct vsp1_entity *entity)
-{
-	struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+		srcrpf |= (!pipe->bru && pipe->num_inputs == 1)
+			? VI6_WPF_SRCRPF_RPF_ACT_MST(input->entity.index)
+			: VI6_WPF_SRCRPF_RPF_ACT_SUB(input->entity.index);
+	}
 
-	vsp1_dlm_destroy(wpf->dlm);
-}
+	if (pipe->bru || pipe->num_inputs > 1)
+		srcrpf |= VI6_WPF_SRCRPF_VIRACT_MST;
 
-static void wpf_set_memory(struct vsp1_entity *entity)
-{
-	struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
+	vsp1_wpf_write(wpf, VI6_WPF_SRCRPF, srcrpf);
 
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, wpf->mem.addr[0]);
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, wpf->mem.addr[1]);
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
+	/* Enable interrupts */
+	vsp1_mod_write(&wpf->entity, VI6_WPF_IRQ_STA(wpf->entity.index), 0);
+	vsp1_mod_write(&wpf->entity, VI6_WPF_IRQ_ENB(wpf->entity.index),
+		       VI6_WFP_IRQ_ENB_FREE);
 }
 
 static const struct vsp1_entity_operations wpf_entity_ops = {
 	.destroy = vsp1_wpf_destroy,
 	.set_memory = wpf_set_memory,
+	.configure = wpf_configure,
 };
 
 /* -----------------------------------------------------------------------------
-- 
2.7.3

[PATCH v2 33/54] v4l: vsp1: Merge RPF and WPF pad ops structures

[Laurent Pinchart]

The two structures are identical, merge them and move the result to
vsp1_rwpf.c. All rwpf pad operations can now be declared static.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_rpf.c  | 12 +------
 drivers/media/platform/vsp1/vsp1_rwpf.c | 60 +++++++++++++++++++--------------
 drivers/media/platform/vsp1/vsp1_rwpf.h | 19 +----------
 drivers/media/platform/vsp1/vsp1_wpf.c  | 12 +------
 4 files changed, 38 insertions(+), 65 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index acc88b4a449b..886312de9089 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -36,18 +36,8 @@ static inline void vsp1_rpf_write(struct vsp1_rwpf *rpf, u32 reg, u32 data)
  * V4L2 Subdevice Operations
  */
 
-static struct v4l2_subdev_pad_ops rpf_pad_ops = {
-	.init_cfg = vsp1_entity_init_cfg,
-	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
-	.enum_frame_size = vsp1_rwpf_enum_frame_size,
-	.get_fmt = vsp1_rwpf_get_format,
-	.set_fmt = vsp1_rwpf_set_format,
-	.get_selection = vsp1_rwpf_get_selection,
-	.set_selection = vsp1_rwpf_set_selection,
-};
-
 static struct v4l2_subdev_ops rpf_ops = {
-	.pad    = &rpf_pad_ops,
+	.pad    = &vsp1_rwpf_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 0c5ad023adfb..4d302f5cccb2 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -20,13 +20,20 @@
 #define RWPF_MIN_WIDTH				1
 #define RWPF_MIN_HEIGHT				1
 
+struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
+				     struct v4l2_subdev_pad_config *config)
+{
+	return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, config,
+					RWPF_PAD_SINK);
+}
+
 /* -----------------------------------------------------------------------------
  * V4L2 Subdevice Pad Operations
  */
 
-int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
-			     struct v4l2_subdev_pad_config *cfg,
-			     struct v4l2_subdev_mbus_code_enum *code)
+static int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
+				    struct v4l2_subdev_pad_config *cfg,
+				    struct v4l2_subdev_mbus_code_enum *code)
 {
 	static const unsigned int codes[] = {
 		MEDIA_BUS_FMT_ARGB8888_1X32,
@@ -41,9 +48,9 @@ int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
-			      struct v4l2_subdev_pad_config *cfg,
-			      struct v4l2_subdev_frame_size_enum *fse)
+static int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
+				     struct v4l2_subdev_pad_config *cfg,
+				     struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
 	struct v4l2_subdev_pad_config *config;
@@ -76,16 +83,9 @@ int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
-				     struct v4l2_subdev_pad_config *config)
-{
-	return v4l2_subdev_get_try_crop(&rwpf->entity.subdev, config,
-					RWPF_PAD_SINK);
-}
-
-int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
-			 struct v4l2_subdev_pad_config *cfg,
-			 struct v4l2_subdev_format *fmt)
+static int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
+				struct v4l2_subdev_pad_config *cfg,
+				struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
 	struct v4l2_subdev_pad_config *config;
@@ -100,9 +100,9 @@ int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
-			 struct v4l2_subdev_pad_config *cfg,
-			 struct v4l2_subdev_format *fmt)
+static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
+				struct v4l2_subdev_pad_config *cfg,
+				struct v4l2_subdev_format *fmt)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
 	struct v4l2_subdev_pad_config *config;
@@ -154,9 +154,9 @@ int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
-			    struct v4l2_subdev_pad_config *cfg,
-			    struct v4l2_subdev_selection *sel)
+static int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
+				   struct v4l2_subdev_pad_config *cfg,
+				   struct v4l2_subdev_selection *sel)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
 	struct v4l2_subdev_pad_config *config;
@@ -191,9 +191,9 @@ int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
-			    struct v4l2_subdev_pad_config *cfg,
-			    struct v4l2_subdev_selection *sel)
+static int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
+				   struct v4l2_subdev_pad_config *cfg,
+				   struct v4l2_subdev_selection *sel)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
 	struct v4l2_subdev_pad_config *config;
@@ -250,6 +250,16 @@ int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+const struct v4l2_subdev_pad_ops vsp1_rwpf_pad_ops = {
+	.init_cfg = vsp1_entity_init_cfg,
+	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
+	.enum_frame_size = vsp1_rwpf_enum_frame_size,
+	.get_fmt = vsp1_rwpf_get_format,
+	.set_fmt = vsp1_rwpf_set_format,
+	.get_selection = vsp1_rwpf_get_selection,
+	.set_selection = vsp1_rwpf_set_selection,
+};
+
 /* -----------------------------------------------------------------------------
  * Controls
  */
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 4ebfab61e0ef..9502710977e8 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -68,24 +68,7 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index);
 
 int vsp1_rwpf_init_ctrls(struct vsp1_rwpf *rwpf);
 
-int vsp1_rwpf_enum_mbus_code(struct v4l2_subdev *subdev,
-			     struct v4l2_subdev_pad_config *cfg,
-			     struct v4l2_subdev_mbus_code_enum *code);
-int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
-			      struct v4l2_subdev_pad_config *cfg,
-			      struct v4l2_subdev_frame_size_enum *fse);
-int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
-			 struct v4l2_subdev_pad_config *cfg,
-			 struct v4l2_subdev_format *fmt);
-int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
-			 struct v4l2_subdev_pad_config *cfg,
-			 struct v4l2_subdev_format *fmt);
-int vsp1_rwpf_get_selection(struct v4l2_subdev *subdev,
-			    struct v4l2_subdev_pad_config *cfg,
-			    struct v4l2_subdev_selection *sel);
-int vsp1_rwpf_set_selection(struct v4l2_subdev *subdev,
-			    struct v4l2_subdev_pad_config *cfg,
-			    struct v4l2_subdev_selection *sel);
+extern const struct v4l2_subdev_pad_ops vsp1_rwpf_pad_ops;
 
 struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
 				     struct v4l2_subdev_pad_config *config);
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 65481930b218..783a923fa76c 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -63,19 +63,9 @@ static struct v4l2_subdev_video_ops wpf_video_ops = {
 	.s_stream = wpf_s_stream,
 };
 
-static struct v4l2_subdev_pad_ops wpf_pad_ops = {
-	.init_cfg = vsp1_entity_init_cfg,
-	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
-	.enum_frame_size = vsp1_rwpf_enum_frame_size,
-	.get_fmt = vsp1_rwpf_get_format,
-	.set_fmt = vsp1_rwpf_set_format,
-	.get_selection = vsp1_rwpf_get_selection,
-	.set_selection = vsp1_rwpf_set_selection,
-};
-
 static struct v4l2_subdev_ops wpf_ops = {
 	.video	= &wpf_video_ops,
-	.pad    = &wpf_pad_ops,
+	.pad    = &vsp1_rwpf_pad_ops,
 };
 
 /* -----------------------------------------------------------------------------
-- 
2.7.3

[PATCH v2 34/54] v4l: vsp1: Use __vsp1_video_try_format to initialize format at init time

[Laurent Pinchart]

Reuse the runtime logic to initialize the default format instead of
open-coding it. This ensures coherency between intialization and
runtime.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_video.c | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index a3f1145c8a79..4dcc892977df 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -958,17 +958,10 @@ struct vsp1_video *vsp1_video_create(struct vsp1_device *vsp1,
 		return ERR_PTR(ret);
 
 	/* ... and the format ... */
-	rwpf->fmtinfo = vsp1_get_format_info(VSP1_VIDEO_DEF_FORMAT);
-	rwpf->format.pixelformat = rwpf->fmtinfo->fourcc;
-	rwpf->format.colorspace = V4L2_COLORSPACE_SRGB;
-	rwpf->format.field = V4L2_FIELD_NONE;
+	rwpf->format.pixelformat = VSP1_VIDEO_DEF_FORMAT;
 	rwpf->format.width = VSP1_VIDEO_DEF_WIDTH;
 	rwpf->format.height = VSP1_VIDEO_DEF_HEIGHT;
-	rwpf->format.num_planes = 1;
-	rwpf->format.plane_fmt[0].bytesperline =
-		rwpf->format.width * rwpf->fmtinfo->bpp[0] / 8;
-	rwpf->format.plane_fmt[0].sizeimage =
-		rwpf->format.plane_fmt[0].bytesperline * rwpf->format.height;
+	__vsp1_video_try_format(video, &rwpf->format, &rwpf->fmtinfo);
 
 	/* ... and the video node... */
 	video->video.v4l2_dev = &video->vsp1->v4l2_dev;
-- 
2.7.3

[PATCH v2 35/54] v4l: vsp1: Pass display list explicitly to configure functions

[Laurent Pinchart]

Modules write register values to the active display list pointed to by
the pipeline. In order to support preparing display lists ahead of time,
pass them explicitly to all configuration functions.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 22 ++++++++--------
 drivers/media/platform/vsp1/vsp1_drm.c    | 14 +++++------
 drivers/media/platform/vsp1/vsp1_entity.c | 15 +++--------
 drivers/media/platform/vsp1/vsp1_entity.h | 14 +++++------
 drivers/media/platform/vsp1/vsp1_hsit.c   | 12 +++++----
 drivers/media/platform/vsp1/vsp1_lif.c    | 12 +++++----
 drivers/media/platform/vsp1/vsp1_lut.c    | 10 +++++---
 drivers/media/platform/vsp1/vsp1_pipe.c   |  3 ++-
 drivers/media/platform/vsp1/vsp1_pipe.h   |  1 +
 drivers/media/platform/vsp1/vsp1_rpf.c    | 39 ++++++++++++++--------------
 drivers/media/platform/vsp1/vsp1_rwpf.h   |  8 +++---
 drivers/media/platform/vsp1/vsp1_sru.c    | 14 ++++++-----
 drivers/media/platform/vsp1/vsp1_uds.c    | 23 +++++++++--------
 drivers/media/platform/vsp1/vsp1_uds.h    |  3 ++-
 drivers/media/platform/vsp1/vsp1_video.c  | 11 +++-----
 drivers/media/platform/vsp1/vsp1_wpf.c    | 42 +++++++++++++++----------------
 16 files changed, 125 insertions(+), 118 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index df57ab6d7e3e..9914410362d0 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -18,6 +18,7 @@
 
 #include "vsp1.h"
 #include "vsp1_bru.h"
+#include "vsp1_dl.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
@@ -28,9 +29,10 @@
  * Device Access
  */
 
-static inline void vsp1_bru_write(struct vsp1_bru *bru, u32 reg, u32 data)
+static inline void vsp1_bru_write(struct vsp1_bru *bru, struct vsp1_dl_list *dl,
+				  u32 reg, u32 data)
 {
-	vsp1_mod_write(&bru->entity, reg, data);
+	vsp1_dl_list_write(dl, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
@@ -303,7 +305,7 @@ static struct v4l2_subdev_ops bru_ops = {
  * VSP1 Entity Operations
  */
 
-static void bru_configure(struct vsp1_entity *entity)
+static void bru_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
 	struct vsp1_bru *bru = to_bru(&entity->subdev);
@@ -324,26 +326,26 @@ static void bru_configure(struct vsp1_entity *entity)
 	 * format at the pipeline output is premultiplied.
 	 */
 	flags = pipe->output ? pipe->output->format.flags : 0;
-	vsp1_bru_write(bru, VI6_BRU_INCTRL,
+	vsp1_bru_write(bru, dl, VI6_BRU_INCTRL,
 		       flags & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA ?
 		       0 : VI6_BRU_INCTRL_NRM);
 
 	/* Set the background position to cover the whole output image and
 	 * configure its color.
 	 */
-	vsp1_bru_write(bru, VI6_BRU_VIRRPF_SIZE,
+	vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_SIZE,
 		       (format->width << VI6_BRU_VIRRPF_SIZE_HSIZE_SHIFT) |
 		       (format->height << VI6_BRU_VIRRPF_SIZE_VSIZE_SHIFT));
-	vsp1_bru_write(bru, VI6_BRU_VIRRPF_LOC, 0);
+	vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_LOC, 0);
 
-	vsp1_bru_write(bru, VI6_BRU_VIRRPF_COL, bru->bgcolor |
+	vsp1_bru_write(bru, dl, VI6_BRU_VIRRPF_COL, bru->bgcolor |
 		       (0xff << VI6_BRU_VIRRPF_COL_A_SHIFT));
 
 	/* Route BRU input 1 as SRC input to the ROP unit and configure the ROP
 	 * unit with a NOP operation to make BRU input 1 available as the
 	 * Blend/ROP unit B SRC input.
 	 */
-	vsp1_bru_write(bru, VI6_BRU_ROP, VI6_BRU_ROP_DSTSEL_BRUIN(1) |
+	vsp1_bru_write(bru, dl, VI6_BRU_ROP, VI6_BRU_ROP_DSTSEL_BRUIN(1) |
 		       VI6_BRU_ROP_CROP(VI6_ROP_NOP) |
 		       VI6_BRU_ROP_AROP(VI6_ROP_NOP));
 
@@ -380,7 +382,7 @@ static void bru_configure(struct vsp1_entity *entity)
 		if (i != 1)
 			ctrl |= VI6_BRU_CTRL_SRCSEL_BRUIN(i);
 
-		vsp1_bru_write(bru, VI6_BRU_CTRL(i), ctrl);
+		vsp1_bru_write(bru, dl, VI6_BRU_CTRL(i), ctrl);
 
 		/* Harcode the blending formula to
 		 *
@@ -394,7 +396,7 @@ static void bru_configure(struct vsp1_entity *entity)
 		 *
 		 * otherwise.
 		 */
-		vsp1_bru_write(bru, VI6_BRU_BLD(i),
+		vsp1_bru_write(bru, dl, VI6_BRU_BLD(i),
 			       VI6_BRU_BLD_CCMDX_255_SRC_A |
 			       (premultiplied ? VI6_BRU_BLD_CCMDY_COEFY :
 						VI6_BRU_BLD_CCMDY_SRC_A) |
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index bec7a651d152..a9735b199a4b 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -455,24 +455,22 @@ void vsp1_du_atomic_flush(struct device *dev)
 			struct vsp1_rwpf *rpf = to_rwpf(&entity->subdev);
 
 			if (!pipe->inputs[rpf->entity.index]) {
-				vsp1_mod_write(entity, entity->route->reg,
-					   VI6_DPR_NODE_UNUSED);
+				vsp1_dl_list_write(pipe->dl, entity->route->reg,
+						   VI6_DPR_NODE_UNUSED);
 				continue;
 			}
 		}
 
-		vsp1_entity_route_setup(entity);
+		vsp1_entity_route_setup(entity, pipe->dl);
 
 		if (entity->ops->configure)
-			entity->ops->configure(entity);
+			entity->ops->configure(entity, pipe->dl);
 
 		if (entity->type == VSP1_ENTITY_RPF)
-			vsp1_rwpf_set_memory(to_rwpf(&entity->subdev));
+			vsp1_rwpf_set_memory(to_rwpf(&entity->subdev),
+					     pipe->dl);
 	}
 
-	/* We know that the WPF s_stream operation never fails. */
-	v4l2_subdev_call(&pipe->output->entity.subdev, video, s_stream, 1);
-
 	vsp1_dl_list_commit(pipe->dl);
 	pipe->dl = NULL;
 
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 37519acd1691..36d425234cd3 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -21,16 +21,9 @@
 #include "vsp1.h"
 #include "vsp1_dl.h"
 #include "vsp1_entity.h"
-#include "vsp1_pipe.h"
 
-void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data)
-{
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&e->subdev.entity);
-
-	vsp1_dl_list_write(pipe->dl, reg, data);
-}
-
-void vsp1_entity_route_setup(struct vsp1_entity *source)
+void vsp1_entity_route_setup(struct vsp1_entity *source,
+			     struct vsp1_dl_list *dl)
 {
 	struct vsp1_entity *sink;
 
@@ -38,8 +31,8 @@ void vsp1_entity_route_setup(struct vsp1_entity *source)
 		return;
 
 	sink = container_of(source->sink, struct vsp1_entity, subdev.entity);
-	vsp1_mod_write(source, source->route->reg,
-		       sink->route->inputs[source->sink_pad]);
+	vsp1_dl_list_write(dl, source->route->reg,
+			   sink->route->inputs[source->sink_pad]);
 }
 
 /* -----------------------------------------------------------------------------
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 3ce74c6f12e1..1db0311b217c 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -19,6 +19,7 @@
 #include <media/v4l2-subdev.h>
 
 struct vsp1_device;
+struct vsp1_dl_list;
 
 enum vsp1_entity_type {
 	VSP1_ENTITY_BRU,
@@ -57,15 +58,15 @@ struct vsp1_route {
  * struct vsp1_entity_operations - Entity operations
  * @destroy:	Destroy the entity.
  * @set_memory:	Setup memory buffer access. This operation applies the settings
- *		stored in the rwpf mem field to the hardware. Valid for RPF and
- *		WPF only.
+ *		stored in the rwpf mem field to the display list. Valid for RPF
+ *		and WPF only.
  * @configure:	Setup the hardware based on the entity state (pipeline, formats,
  *		selection rectangles, ...)
  */
 struct vsp1_entity_operations {
 	void (*destroy)(struct vsp1_entity *);
-	void (*set_memory)(struct vsp1_entity *);
-	void (*configure)(struct vsp1_entity *);
+	void (*set_memory)(struct vsp1_entity *, struct vsp1_dl_list *dl);
+	void (*configure)(struct vsp1_entity *, struct vsp1_dl_list *dl);
 };
 
 struct vsp1_entity {
@@ -121,8 +122,7 @@ vsp1_entity_get_pad_compose(struct vsp1_entity *entity,
 int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
 			 struct v4l2_subdev_pad_config *cfg);
 
-void vsp1_entity_route_setup(struct vsp1_entity *source);
-
-void vsp1_mod_write(struct vsp1_entity *e, u32 reg, u32 data);
+void vsp1_entity_route_setup(struct vsp1_entity *source,
+			     struct vsp1_dl_list *dl);
 
 #endif /* __VSP1_ENTITY_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index b935a62e6399..6352a9e32983 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -17,6 +17,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_hsit.h"
 
 #define HSIT_MIN_SIZE				4U
@@ -26,9 +27,10 @@
  * Device Access
  */
 
-static inline void vsp1_hsit_write(struct vsp1_hsit *hsit, u32 reg, u32 data)
+static inline void vsp1_hsit_write(struct vsp1_hsit *hsit,
+				   struct vsp1_dl_list *dl, u32 reg, u32 data)
 {
-	vsp1_mod_write(&hsit->entity, reg, data);
+	vsp1_dl_list_write(dl, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
@@ -164,14 +166,14 @@ static struct v4l2_subdev_ops hsit_ops = {
  * VSP1 Entity Operations
  */
 
-static void hsit_configure(struct vsp1_entity *entity)
+static void hsit_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_hsit *hsit = to_hsit(&entity->subdev);
 
 	if (hsit->inverse)
-		vsp1_hsit_write(hsit, VI6_HSI_CTRL, VI6_HSI_CTRL_EN);
+		vsp1_hsit_write(hsit, dl, VI6_HSI_CTRL, VI6_HSI_CTRL_EN);
 	else
-		vsp1_hsit_write(hsit, VI6_HST_CTRL, VI6_HST_CTRL_EN);
+		vsp1_hsit_write(hsit, dl, VI6_HST_CTRL, VI6_HST_CTRL_EN);
 }
 
 static const struct vsp1_entity_operations hsit_entity_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index d3b94055f23c..ba8610fa45a0 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -17,6 +17,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_lif.h"
 
 #define LIF_MIN_SIZE				2U
@@ -26,9 +27,10 @@
  * Device Access
  */
 
-static inline void vsp1_lif_write(struct vsp1_lif *lif, u32 reg, u32 data)
+static inline void vsp1_lif_write(struct vsp1_lif *lif, struct vsp1_dl_list *dl,
+				  u32 reg, u32 data)
 {
-	vsp1_mod_write(&lif->entity, reg, data);
+	vsp1_dl_list_write(dl, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
@@ -183,7 +185,7 @@ static struct v4l2_subdev_ops lif_ops = {
  * VSP1 Entity Operations
  */
 
-static void lif_configure(struct vsp1_entity *entity)
+static void lif_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	const struct v4l2_mbus_framefmt *format;
 	struct vsp1_lif *lif = to_lif(&entity->subdev);
@@ -196,11 +198,11 @@ static void lif_configure(struct vsp1_entity *entity)
 
 	obth = min(obth, (format->width + 1) / 2 * format->height - 4);
 
-	vsp1_lif_write(lif, VI6_LIF_CSBTH,
+	vsp1_lif_write(lif, dl, VI6_LIF_CSBTH,
 			(hbth << VI6_LIF_CSBTH_HBTH_SHIFT) |
 			(lbth << VI6_LIF_CSBTH_LBTH_SHIFT));
 
-	vsp1_lif_write(lif, VI6_LIF_CTRL,
+	vsp1_lif_write(lif, dl, VI6_LIF_CTRL,
 			(obth << VI6_LIF_CTRL_OBTH_SHIFT) |
 			(format->code == 0 ? VI6_LIF_CTRL_CFMT : 0) |
 			VI6_LIF_CTRL_REQSEL | VI6_LIF_CTRL_LIF_EN);
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 1e8d43460d49..93c6b45f96ad 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -18,6 +18,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_lut.h"
 
 #define LUT_MIN_SIZE				4U
@@ -27,9 +28,10 @@
  * Device Access
  */
 
-static inline void vsp1_lut_write(struct vsp1_lut *lut, u32 reg, u32 data)
+static inline void vsp1_lut_write(struct vsp1_lut *lut, struct vsp1_dl_list *dl,
+				  u32 reg, u32 data)
 {
-	vsp1_mod_write(&lut->entity, reg, data);
+	vsp1_dl_list_write(dl, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
@@ -219,11 +221,11 @@ static struct v4l2_subdev_ops lut_ops = {
  * VSP1 Entity Operations
  */
 
-static void lut_configure(struct vsp1_entity *entity)
+static void lut_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_lut *lut = to_lut(&entity->subdev);
 
-	vsp1_lut_write(lut, VI6_LUT_CTRL, VI6_LUT_CTRL_EN);
+	vsp1_lut_write(lut, dl, VI6_LUT_CTRL, VI6_LUT_CTRL_EN);
 }
 
 static const struct vsp1_entity_operations lut_entity_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index fe2538d5bed1..4d06519f717d 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -295,6 +295,7 @@ void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe)
  */
 void vsp1_pipeline_propagate_alpha(struct vsp1_pipeline *pipe,
 				   struct vsp1_entity *input,
+				   struct vsp1_dl_list *dl,
 				   unsigned int alpha)
 {
 	struct vsp1_entity *entity;
@@ -317,7 +318,7 @@ void vsp1_pipeline_propagate_alpha(struct vsp1_pipeline *pipe,
 		if (entity->type == VSP1_ENTITY_UDS) {
 			struct vsp1_uds *uds = to_uds(&entity->subdev);
 
-			vsp1_uds_set_alpha(uds, alpha);
+			vsp1_uds_set_alpha(uds, dl, alpha);
 			break;
 		}
 
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h b/drivers/media/platform/vsp1/vsp1_pipe.h
index f4bdfc943add..1100229a1ed2 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -123,6 +123,7 @@ void vsp1_pipeline_frame_end(struct vsp1_pipeline *pipe);
 
 void vsp1_pipeline_propagate_alpha(struct vsp1_pipeline *pipe,
 				   struct vsp1_entity *input,
+				   struct vsp1_dl_list *dl,
 				   unsigned int alpha);
 
 void vsp1_pipelines_suspend(struct vsp1_device *vsp1);
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 886312de9089..c8127f2999d2 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -16,6 +16,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
@@ -26,10 +27,10 @@
  * Device Access
  */
 
-static inline void vsp1_rpf_write(struct vsp1_rwpf *rpf, u32 reg, u32 data)
+static inline void vsp1_rpf_write(struct vsp1_rwpf *rpf,
+				  struct vsp1_dl_list *dl, u32 reg, u32 data)
 {
-	vsp1_mod_write(&rpf->entity, reg + rpf->entity.index * VI6_RPF_OFFSET,
-		       data);
+	vsp1_dl_list_write(dl, reg + rpf->entity.index * VI6_RPF_OFFSET, data);
 }
 
 /* -----------------------------------------------------------------------------
@@ -44,19 +45,19 @@ static struct v4l2_subdev_ops rpf_ops = {
  * VSP1 Entity Operations
  */
 
-static void rpf_set_memory(struct vsp1_entity *entity)
+static void rpf_set_memory(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_rwpf *rpf = entity_to_rwpf(entity);
 
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_Y,
+	vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_ADDR_Y,
 		       rpf->mem.addr[0] + rpf->offsets[0]);
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C0,
+	vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_ADDR_C0,
 		       rpf->mem.addr[1] + rpf->offsets[1]);
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_ADDR_C1,
+	vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_ADDR_C1,
 		       rpf->mem.addr[2] + rpf->offsets[1]);
 }
 
-static void rpf_configure(struct vsp1_entity *entity)
+static void rpf_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
 	struct vsp1_rwpf *rpf = to_rwpf(&entity->subdev);
@@ -78,10 +79,10 @@ static void rpf_configure(struct vsp1_entity *entity)
 	 */
 	crop = vsp1_rwpf_get_crop(rpf, rpf->entity.config);
 
-	vsp1_rpf_write(rpf, VI6_RPF_SRC_BSIZE,
+	vsp1_rpf_write(rpf, dl, VI6_RPF_SRC_BSIZE,
 		       (crop->width << VI6_RPF_SRC_BSIZE_BHSIZE_SHIFT) |
 		       (crop->height << VI6_RPF_SRC_BSIZE_BVSIZE_SHIFT));
-	vsp1_rpf_write(rpf, VI6_RPF_SRC_ESIZE,
+	vsp1_rpf_write(rpf, dl, VI6_RPF_SRC_ESIZE,
 		       (crop->width << VI6_RPF_SRC_ESIZE_EHSIZE_SHIFT) |
 		       (crop->height << VI6_RPF_SRC_ESIZE_EVSIZE_SHIFT));
 
@@ -99,7 +100,7 @@ static void rpf_configure(struct vsp1_entity *entity)
 		rpf->offsets[1] = 0;
 	}
 
-	vsp1_rpf_write(rpf, VI6_RPF_SRCM_PSTRIDE, pstride);
+	vsp1_rpf_write(rpf, dl, VI6_RPF_SRCM_PSTRIDE, pstride);
 
 	/* Format */
 	sink_format = vsp1_entity_get_pad_format(&rpf->entity,
@@ -120,8 +121,8 @@ static void rpf_configure(struct vsp1_entity *entity)
 	if (sink_format->code != source_format->code)
 		infmt |= VI6_RPF_INFMT_CSC;
 
-	vsp1_rpf_write(rpf, VI6_RPF_INFMT, infmt);
-	vsp1_rpf_write(rpf, VI6_RPF_DSWAP, fmtinfo->swap);
+	vsp1_rpf_write(rpf, dl, VI6_RPF_INFMT, infmt);
+	vsp1_rpf_write(rpf, dl, VI6_RPF_DSWAP, fmtinfo->swap);
 
 	/* Output location */
 	if (pipe->bru) {
@@ -134,7 +135,7 @@ static void rpf_configure(struct vsp1_entity *entity)
 		top = compose->top;
 	}
 
-	vsp1_rpf_write(rpf, VI6_RPF_LOC,
+	vsp1_rpf_write(rpf, dl, VI6_RPF_LOC,
 		       (left << VI6_RPF_LOC_HCOORD_SHIFT) |
 		       (top << VI6_RPF_LOC_VCOORD_SHIFT));
 
@@ -142,17 +143,17 @@ static void rpf_configure(struct vsp1_entity *entity)
 	 * alpha value set through the V4L2_CID_ALPHA_COMPONENT control
 	 * otherwise. Disable color keying.
 	 */
-	vsp1_rpf_write(rpf, VI6_RPF_ALPH_SEL, VI6_RPF_ALPH_SEL_AEXT_EXT |
+	vsp1_rpf_write(rpf, dl, VI6_RPF_ALPH_SEL, VI6_RPF_ALPH_SEL_AEXT_EXT |
 		       (fmtinfo->alpha ? VI6_RPF_ALPH_SEL_ASEL_PACKED
 				       : VI6_RPF_ALPH_SEL_ASEL_FIXED));
 
-	vsp1_rpf_write(rpf, VI6_RPF_VRTCOL_SET,
+	vsp1_rpf_write(rpf, dl, VI6_RPF_VRTCOL_SET,
 		       rpf->alpha << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
 
-	vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, rpf->alpha);
+	vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, dl, rpf->alpha);
 
-	vsp1_rpf_write(rpf, VI6_RPF_MSK_CTRL, 0);
-	vsp1_rpf_write(rpf, VI6_RPF_CKEY_CTRL, 0);
+	vsp1_rpf_write(rpf, dl, VI6_RPF_MSK_CTRL, 0);
+	vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL, 0);
 }
 
 static const struct vsp1_entity_operations rpf_entity_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 9502710977e8..38c8c902db52 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -75,12 +75,14 @@ struct v4l2_rect *vsp1_rwpf_get_crop(struct vsp1_rwpf *rwpf,
 /**
  * vsp1_rwpf_set_memory - Configure DMA addresses for a [RW]PF
  * @rwpf: the [RW]PF instance
+ * @dl: the display list
  *
- * This function applies the cached memory buffer address to the hardware.
+ * This function applies the cached memory buffer address to the display list.
  */
-static inline void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf)
+static inline void vsp1_rwpf_set_memory(struct vsp1_rwpf *rwpf,
+					struct vsp1_dl_list *dl)
 {
-	rwpf->entity.ops->set_memory(&rwpf->entity);
+	rwpf->entity.ops->set_memory(&rwpf->entity, dl);
 }
 
 #endif /* __VSP1_RWPF_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 82a8ee202f1a..bb3389dda9c5 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -17,6 +17,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_sru.h"
 
 #define SRU_MIN_SIZE				4U
@@ -26,9 +27,10 @@
  * Device Access
  */
 
-static inline void vsp1_sru_write(struct vsp1_sru *sru, u32 reg, u32 data)
+static inline void vsp1_sru_write(struct vsp1_sru *sru, struct vsp1_dl_list *dl,
+				  u32 reg, u32 data)
 {
-	vsp1_mod_write(&sru->entity, reg, data);
+	vsp1_dl_list_write(dl, reg, data);
 }
 
 /* -----------------------------------------------------------------------------
@@ -295,7 +297,7 @@ static struct v4l2_subdev_ops sru_ops = {
  * VSP1 Entity Operations
  */
 
-static void sru_configure(struct vsp1_entity *entity)
+static void sru_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	const struct vsp1_sru_param *param;
 	struct vsp1_sru *sru = to_sru(&entity->subdev);
@@ -321,9 +323,9 @@ static void sru_configure(struct vsp1_entity *entity)
 
 	ctrl0 |= param->ctrl0;
 
-	vsp1_sru_write(sru, VI6_SRU_CTRL0, ctrl0);
-	vsp1_sru_write(sru, VI6_SRU_CTRL1, VI6_SRU_CTRL1_PARAM5);
-	vsp1_sru_write(sru, VI6_SRU_CTRL2, param->ctrl2);
+	vsp1_sru_write(sru, dl, VI6_SRU_CTRL0, ctrl0);
+	vsp1_sru_write(sru, dl, VI6_SRU_CTRL1, VI6_SRU_CTRL1_PARAM5);
+	vsp1_sru_write(sru, dl, VI6_SRU_CTRL2, param->ctrl2);
 }
 
 static const struct vsp1_entity_operations sru_entity_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 88c44f450f88..cdc033050a9b 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -17,6 +17,7 @@
 #include <media/v4l2-subdev.h>
 
 #include "vsp1.h"
+#include "vsp1_dl.h"
 #include "vsp1_uds.h"
 
 #define UDS_MIN_SIZE				4U
@@ -29,19 +30,21 @@
  * Device Access
  */
 
-static inline void vsp1_uds_write(struct vsp1_uds *uds, u32 reg, u32 data)
+static inline void vsp1_uds_write(struct vsp1_uds *uds, struct vsp1_dl_list *dl,
+				  u32 reg, u32 data)
 {
-	vsp1_mod_write(&uds->entity, reg + uds->entity.index * VI6_UDS_OFFSET,
-		       data);
+	vsp1_dl_list_write(dl, reg + uds->entity.index * VI6_UDS_OFFSET, data);
 }
 
 /* -----------------------------------------------------------------------------
  * Scaling Computation
  */
 
-void vsp1_uds_set_alpha(struct vsp1_uds *uds, unsigned int alpha)
+void vsp1_uds_set_alpha(struct vsp1_uds *uds, struct vsp1_dl_list *dl,
+			unsigned int alpha)
 {
-	vsp1_uds_write(uds, VI6_UDS_ALPVAL, alpha << VI6_UDS_ALPVAL_VAL0_SHIFT);
+	vsp1_uds_write(uds, dl, VI6_UDS_ALPVAL,
+		       alpha << VI6_UDS_ALPVAL_VAL0_SHIFT);
 }
 
 /*
@@ -281,7 +284,7 @@ static struct v4l2_subdev_ops uds_ops = {
  * VSP1 Entity Operations
  */
 
-static void uds_configure(struct vsp1_entity *entity)
+static void uds_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_uds *uds = to_uds(&entity->subdev);
 	const struct v4l2_mbus_framefmt *output;
@@ -309,21 +312,21 @@ static void uds_configure(struct vsp1_entity *entity)
 	else
 		multitap = true;
 
-	vsp1_uds_write(uds, VI6_UDS_CTRL,
+	vsp1_uds_write(uds, dl, VI6_UDS_CTRL,
 		       (uds->scale_alpha ? VI6_UDS_CTRL_AON : 0) |
 		       (multitap ? VI6_UDS_CTRL_BC : 0));
 
-	vsp1_uds_write(uds, VI6_UDS_PASS_BWIDTH,
+	vsp1_uds_write(uds, dl, VI6_UDS_PASS_BWIDTH,
 		       (uds_passband_width(hscale)
 				<< VI6_UDS_PASS_BWIDTH_H_SHIFT) |
 		       (uds_passband_width(vscale)
 				<< VI6_UDS_PASS_BWIDTH_V_SHIFT));
 
 	/* Set the scaling ratios and the output size. */
-	vsp1_uds_write(uds, VI6_UDS_SCALE,
+	vsp1_uds_write(uds, dl, VI6_UDS_SCALE,
 		       (hscale << VI6_UDS_SCALE_HFRAC_SHIFT) |
 		       (vscale << VI6_UDS_SCALE_VFRAC_SHIFT));
-	vsp1_uds_write(uds, VI6_UDS_CLIP_SIZE,
+	vsp1_uds_write(uds, dl, VI6_UDS_CLIP_SIZE,
 		       (output->width << VI6_UDS_CLIP_SIZE_HSIZE_SHIFT) |
 		       (output->height << VI6_UDS_CLIP_SIZE_VSIZE_SHIFT));
 }
diff --git a/drivers/media/platform/vsp1/vsp1_uds.h b/drivers/media/platform/vsp1/vsp1_uds.h
index 031ac0da1b66..5c8cbfcad4cc 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.h
+++ b/drivers/media/platform/vsp1/vsp1_uds.h
@@ -35,6 +35,7 @@ static inline struct vsp1_uds *to_uds(struct v4l2_subdev *subdev)
 
 struct vsp1_uds *vsp1_uds_create(struct vsp1_device *vsp1, unsigned int index);
 
-void vsp1_uds_set_alpha(struct vsp1_uds *uds, unsigned int alpha);
+void vsp1_uds_set_alpha(struct vsp1_uds *uds, struct vsp1_dl_list *dl,
+			unsigned int alpha);
 
 #endif /* __VSP1_UDS_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 4dcc892977df..a45bf68e0ba1 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -455,11 +455,11 @@ static void vsp1_video_pipeline_run(struct vsp1_pipeline *pipe)
 		struct vsp1_rwpf *rwpf = pipe->inputs[i];
 
 		if (rwpf)
-			vsp1_rwpf_set_memory(rwpf);
+			vsp1_rwpf_set_memory(rwpf, pipe->dl);
 	}
 
 	if (!pipe->lif)
-		vsp1_rwpf_set_memory(pipe->output);
+		vsp1_rwpf_set_memory(pipe->output, pipe->dl);
 
 	vsp1_dl_list_commit(pipe->dl);
 	pipe->dl = NULL;
@@ -616,15 +616,12 @@ static int vsp1_video_setup_pipeline(struct vsp1_pipeline *pipe)
 	}
 
 	list_for_each_entry(entity, &pipe->entities, list_pipe) {
-		vsp1_entity_route_setup(entity);
+		vsp1_entity_route_setup(entity, pipe->dl);
 
 		if (entity->ops->configure)
-			entity->ops->configure(entity);
+			entity->ops->configure(entity, pipe->dl);
 	}
 
-	/* We know that the WPF s_stream operation never fails. */
-	v4l2_subdev_call(&pipe->output->entity.subdev, video, s_stream, 1);
-
 	return 0;
 }
 
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 783a923fa76c..36b4c9fdead6 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -27,10 +27,10 @@
  * Device Access
  */
 
-static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf, u32 reg, u32 data)
+static inline void vsp1_wpf_write(struct vsp1_rwpf *wpf,
+				  struct vsp1_dl_list *dl, u32 reg, u32 data)
 {
-	vsp1_mod_write(&wpf->entity,
-		       reg + wpf->entity.index * VI6_WPF_OFFSET, data);
+	vsp1_dl_list_write(dl, reg + wpf->entity.index * VI6_WPF_OFFSET, data);
 }
 
 /* -----------------------------------------------------------------------------
@@ -79,16 +79,16 @@ static void vsp1_wpf_destroy(struct vsp1_entity *entity)
 	vsp1_dlm_destroy(wpf->dlm);
 }
 
-static void wpf_set_memory(struct vsp1_entity *entity)
+static void wpf_set_memory(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_rwpf *wpf = entity_to_rwpf(entity);
 
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_Y, wpf->mem.addr[0]);
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C0, wpf->mem.addr[1]);
-	vsp1_wpf_write(wpf, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
+	vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_ADDR_Y, wpf->mem.addr[0]);
+	vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_ADDR_C0, wpf->mem.addr[1]);
+	vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
 }
 
-static void wpf_configure(struct vsp1_entity *entity)
+static void wpf_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 {
 	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
 	struct vsp1_rwpf *wpf = to_rwpf(&entity->subdev);
@@ -103,10 +103,10 @@ static void wpf_configure(struct vsp1_entity *entity)
 	/* Cropping */
 	crop = vsp1_rwpf_get_crop(wpf, wpf->entity.config);
 
-	vsp1_wpf_write(wpf, VI6_WPF_HSZCLIP, VI6_WPF_SZCLIP_EN |
+	vsp1_wpf_write(wpf, dl, VI6_WPF_HSZCLIP, VI6_WPF_SZCLIP_EN |
 		       (crop->left << VI6_WPF_SZCLIP_OFST_SHIFT) |
 		       (crop->width << VI6_WPF_SZCLIP_SIZE_SHIFT));
-	vsp1_wpf_write(wpf, VI6_WPF_VSZCLIP, VI6_WPF_SZCLIP_EN |
+	vsp1_wpf_write(wpf, dl, VI6_WPF_VSZCLIP, VI6_WPF_SZCLIP_EN |
 		       (crop->top << VI6_WPF_SZCLIP_OFST_SHIFT) |
 		       (crop->height << VI6_WPF_SZCLIP_SIZE_SHIFT));
 
@@ -132,25 +132,25 @@ static void wpf_configure(struct vsp1_entity *entity)
 			outfmt |= VI6_WPF_OUTFMT_SPUVS;
 
 		/* Destination stride and byte swapping. */
-		vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_Y,
+		vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_STRIDE_Y,
 			       format->plane_fmt[0].bytesperline);
 		if (format->num_planes > 1)
-			vsp1_wpf_write(wpf, VI6_WPF_DSTM_STRIDE_C,
+			vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_STRIDE_C,
 				       format->plane_fmt[1].bytesperline);
 
-		vsp1_wpf_write(wpf, VI6_WPF_DSWAP, fmtinfo->swap);
+		vsp1_wpf_write(wpf, dl, VI6_WPF_DSWAP, fmtinfo->swap);
 	}
 
 	if (sink_format->code != source_format->code)
 		outfmt |= VI6_WPF_OUTFMT_CSC;
 
 	outfmt |= wpf->alpha << VI6_WPF_OUTFMT_PDV_SHIFT;
-	vsp1_wpf_write(wpf, VI6_WPF_OUTFMT, outfmt);
+	vsp1_wpf_write(wpf, dl, VI6_WPF_OUTFMT, outfmt);
 
-	vsp1_mod_write(&wpf->entity, VI6_DPR_WPF_FPORCH(wpf->entity.index),
-		       VI6_DPR_WPF_FPORCH_FP_WPFN);
+	vsp1_dl_list_write(dl, VI6_DPR_WPF_FPORCH(wpf->entity.index),
+			   VI6_DPR_WPF_FPORCH_FP_WPFN);
 
-	vsp1_mod_write(&wpf->entity, VI6_WPF_WRBCK_CTRL, 0);
+	vsp1_dl_list_write(dl, VI6_WPF_WRBCK_CTRL, 0);
 
 	/* Sources. If the pipeline has a single input and BRU is not used,
 	 * configure it as the master layer. Otherwise configure all
@@ -171,12 +171,12 @@ static void wpf_configure(struct vsp1_entity *entity)
 	if (pipe->bru || pipe->num_inputs > 1)
 		srcrpf |= VI6_WPF_SRCRPF_VIRACT_MST;
 
-	vsp1_wpf_write(wpf, VI6_WPF_SRCRPF, srcrpf);
+	vsp1_wpf_write(wpf, dl, VI6_WPF_SRCRPF, srcrpf);
 
 	/* Enable interrupts */
-	vsp1_mod_write(&wpf->entity, VI6_WPF_IRQ_STA(wpf->entity.index), 0);
-	vsp1_mod_write(&wpf->entity, VI6_WPF_IRQ_ENB(wpf->entity.index),
-		       VI6_WFP_IRQ_ENB_FREE);
+	vsp1_dl_list_write(dl, VI6_WPF_IRQ_STA(wpf->entity.index), 0);
+	vsp1_dl_list_write(dl, VI6_WPF_IRQ_ENB(wpf->entity.index),
+			   VI6_WFP_IRQ_ENB_FREE);
 }
 
 static const struct vsp1_entity_operations wpf_entity_ops = {
-- 
2.7.3

[PATCH v2 36/54] v4l: vsp1: Rename pipeline validate functions to pipeline build

[Laurent Pinchart]

The primary purpose of those functions is to build the pipeline, rename
them to make this clearer.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_video.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index a45bf68e0ba1..ddf440dc9a9c 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -172,9 +172,9 @@ static int __vsp1_video_try_format(struct vsp1_video *video,
  * Pipeline Management
  */
 
-static int vsp1_video_pipeline_validate_branch(struct vsp1_pipeline *pipe,
-					       struct vsp1_rwpf *input,
-					       struct vsp1_rwpf *output)
+static int vsp1_video_pipeline_build_branch(struct vsp1_pipeline *pipe,
+					    struct vsp1_rwpf *input,
+					    struct vsp1_rwpf *output)
 {
 	struct media_entity_enum ent_enum;
 	struct vsp1_entity *entity;
@@ -257,8 +257,8 @@ out:
 	return ret;
 }
 
-static int vsp1_video_pipeline_validate(struct vsp1_pipeline *pipe,
-					struct vsp1_video *video)
+static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
+				     struct vsp1_video *video)
 {
 	struct media_entity_graph graph;
 	struct media_entity *entity = &video->video.entity;
@@ -321,8 +321,8 @@ static int vsp1_video_pipeline_validate(struct vsp1_pipeline *pipe,
 		if (!pipe->inputs[i])
 			continue;
 
-		ret = vsp1_video_pipeline_validate_branch(pipe, pipe->inputs[i],
-							  pipe->output);
+		ret = vsp1_video_pipeline_build_branch(pipe, pipe->inputs[i],
+						       pipe->output);
 		if (ret < 0)
 			goto error;
 	}
@@ -341,9 +341,9 @@ static int vsp1_video_pipeline_init(struct vsp1_pipeline *pipe,
 
 	mutex_lock(&pipe->lock);
 
-	/* If we're the first user validate and initialize the pipeline. */
+	/* If we're the first user build and validate the pipeline. */
 	if (pipe->use_count == 0) {
-		ret = vsp1_video_pipeline_validate(pipe, video);
+		ret = vsp1_video_pipeline_build(pipe, video);
 		if (ret < 0)
 			goto done;
 	}
-- 
2.7.3

[PATCH v2 37/54] v4l: vsp1: Pass pipe pointer to entity configure functions

[Laurent Pinchart]

Pass the pipe explicitly instead of retrieving it through media
entities. This decouples device state stored in the pipeline from the
active state stored in entities, preparing for dynamic pipeline
creation.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 5 +++--
 drivers/media/platform/vsp1/vsp1_drm.c    | 2 +-
 drivers/media/platform/vsp1/vsp1_entity.h | 4 +++-
 drivers/media/platform/vsp1/vsp1_hsit.c   | 4 +++-
 drivers/media/platform/vsp1/vsp1_lif.c    | 4 +++-
 drivers/media/platform/vsp1/vsp1_lut.c    | 4 +++-
 drivers/media/platform/vsp1/vsp1_rpf.c    | 5 +++--
 drivers/media/platform/vsp1/vsp1_sru.c    | 4 +++-
 drivers/media/platform/vsp1/vsp1_uds.c    | 4 +++-
 drivers/media/platform/vsp1/vsp1_video.c  | 2 +-
 drivers/media/platform/vsp1/vsp1_wpf.c    | 5 +++--
 11 files changed, 29 insertions(+), 14 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 9914410362d0..820c3c90a4a6 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -305,9 +305,10 @@ static struct v4l2_subdev_ops bru_ops = {
  * VSP1 Entity Operations
  */
 
-static void bru_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void bru_configure(struct vsp1_entity *entity,
+			  struct vsp1_pipeline *pipe,
+			  struct vsp1_dl_list *dl)
 {
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
 	struct vsp1_bru *bru = to_bru(&entity->subdev);
 	struct v4l2_mbus_framefmt *format;
 	unsigned int flags;
diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index a9735b199a4b..7cde2d970dba 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -464,7 +464,7 @@ void vsp1_du_atomic_flush(struct device *dev)
 		vsp1_entity_route_setup(entity, pipe->dl);
 
 		if (entity->ops->configure)
-			entity->ops->configure(entity, pipe->dl);
+			entity->ops->configure(entity, pipe, pipe->dl);
 
 		if (entity->type == VSP1_ENTITY_RPF)
 			vsp1_rwpf_set_memory(to_rwpf(&entity->subdev),
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 1db0311b217c..b58bcb15c087 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -20,6 +20,7 @@
 
 struct vsp1_device;
 struct vsp1_dl_list;
+struct vsp1_pipeline;
 
 enum vsp1_entity_type {
 	VSP1_ENTITY_BRU,
@@ -66,7 +67,8 @@ struct vsp1_route {
 struct vsp1_entity_operations {
 	void (*destroy)(struct vsp1_entity *);
 	void (*set_memory)(struct vsp1_entity *, struct vsp1_dl_list *dl);
-	void (*configure)(struct vsp1_entity *, struct vsp1_dl_list *dl);
+	void (*configure)(struct vsp1_entity *, struct vsp1_pipeline *,
+			  struct vsp1_dl_list *);
 };
 
 struct vsp1_entity {
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index 6352a9e32983..1b67ba9a69ed 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -166,7 +166,9 @@ static struct v4l2_subdev_ops hsit_ops = {
  * VSP1 Entity Operations
  */
 
-static void hsit_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void hsit_configure(struct vsp1_entity *entity,
+			   struct vsp1_pipeline *pipe,
+			   struct vsp1_dl_list *dl)
 {
 	struct vsp1_hsit *hsit = to_hsit(&entity->subdev);
 
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index ba8610fa45a0..628109aba23b 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -185,7 +185,9 @@ static struct v4l2_subdev_ops lif_ops = {
  * VSP1 Entity Operations
  */
 
-static void lif_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void lif_configure(struct vsp1_entity *entity,
+			  struct vsp1_pipeline *pipe,
+			  struct vsp1_dl_list *dl)
 {
 	const struct v4l2_mbus_framefmt *format;
 	struct vsp1_lif *lif = to_lif(&entity->subdev);
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 93c6b45f96ad..fba6cc4e67a1 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -221,7 +221,9 @@ static struct v4l2_subdev_ops lut_ops = {
  * VSP1 Entity Operations
  */
 
-static void lut_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void lut_configure(struct vsp1_entity *entity,
+			  struct vsp1_pipeline *pipe,
+			  struct vsp1_dl_list *dl)
 {
 	struct vsp1_lut *lut = to_lut(&entity->subdev);
 
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index c8127f2999d2..e373f1473c1f 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -57,9 +57,10 @@ static void rpf_set_memory(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 		       rpf->mem.addr[2] + rpf->offsets[1]);
 }
 
-static void rpf_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void rpf_configure(struct vsp1_entity *entity,
+			  struct vsp1_pipeline *pipe,
+			  struct vsp1_dl_list *dl)
 {
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
 	struct vsp1_rwpf *rpf = to_rwpf(&entity->subdev);
 	const struct vsp1_format_info *fmtinfo = rpf->fmtinfo;
 	const struct v4l2_pix_format_mplane *format = &rpf->format;
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index bb3389dda9c5..3dc335d1499d 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -297,7 +297,9 @@ static struct v4l2_subdev_ops sru_ops = {
  * VSP1 Entity Operations
  */
 
-static void sru_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void sru_configure(struct vsp1_entity *entity,
+			  struct vsp1_pipeline *pipe,
+			  struct vsp1_dl_list *dl)
 {
 	const struct vsp1_sru_param *param;
 	struct vsp1_sru *sru = to_sru(&entity->subdev);
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index cdc033050a9b..0bc0616f6b18 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -284,7 +284,9 @@ static struct v4l2_subdev_ops uds_ops = {
  * VSP1 Entity Operations
  */
 
-static void uds_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void uds_configure(struct vsp1_entity *entity,
+			  struct vsp1_pipeline *pipe,
+			  struct vsp1_dl_list *dl)
 {
 	struct vsp1_uds *uds = to_uds(&entity->subdev);
 	const struct v4l2_mbus_framefmt *output;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index ddf440dc9a9c..a16a661e5b69 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -619,7 +619,7 @@ static int vsp1_video_setup_pipeline(struct vsp1_pipeline *pipe)
 		vsp1_entity_route_setup(entity, pipe->dl);
 
 		if (entity->ops->configure)
-			entity->ops->configure(entity, pipe->dl);
+			entity->ops->configure(entity, pipe, pipe->dl);
 	}
 
 	return 0;
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 36b4c9fdead6..5aa5eb4b8121 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -88,9 +88,10 @@ static void wpf_set_memory(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
 	vsp1_wpf_write(wpf, dl, VI6_WPF_DSTM_ADDR_C1, wpf->mem.addr[2]);
 }
 
-static void wpf_configure(struct vsp1_entity *entity, struct vsp1_dl_list *dl)
+static void wpf_configure(struct vsp1_entity *entity,
+			  struct vsp1_pipeline *pipe,
+			  struct vsp1_dl_list *dl)
 {
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&entity->subdev.entity);
 	struct vsp1_rwpf *wpf = to_rwpf(&entity->subdev);
 	struct vsp1_device *vsp1 = wpf->entity.vsp1;
 	const struct v4l2_mbus_framefmt *source_format;
-- 
2.7.3

[PATCH v2 38/54] v4l: vsp1: Store pipeline pointer in rwpf

[Laurent Pinchart]

This prepares for dynamic pipeline allocation by providing a field that
can be used to store the pipeline pointer atomically under driver
control.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_drv.c   |  4 +---
 drivers/media/platform/vsp1/vsp1_pipe.c  | 14 +++++++++-----
 drivers/media/platform/vsp1/vsp1_pipe.h  |  8 --------
 drivers/media/platform/vsp1/vsp1_rwpf.h  |  2 ++
 drivers/media/platform/vsp1/vsp1_video.c | 13 +++++++------
 5 files changed, 19 insertions(+), 22 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index bfdc01c9172d..f1be2680013d 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -49,17 +49,15 @@ static irqreturn_t vsp1_irq_handler(int irq, void *data)
 
 	for (i = 0; i < vsp1->info->wpf_count; ++i) {
 		struct vsp1_rwpf *wpf = vsp1->wpf[i];
-		struct vsp1_pipeline *pipe;
 
 		if (wpf == NULL)
 			continue;
 
-		pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
 		status = vsp1_read(vsp1, VI6_WPF_IRQ_STA(i));
 		vsp1_write(vsp1, VI6_WPF_IRQ_STA(i), ~status & mask);
 
 		if (status & VI6_WFP_IRQ_STA_FRE) {
-			vsp1_pipeline_frame_end(pipe);
+			vsp1_pipeline_frame_end(wpf->pipe);
 			ret = IRQ_HANDLED;
 		}
 	}
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 4d06519f717d..8ac080f87b08 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -172,14 +172,18 @@ void vsp1_pipeline_reset(struct vsp1_pipeline *pipe)
 			bru->inputs[i].rpf = NULL;
 	}
 
-	for (i = 0; i < ARRAY_SIZE(pipe->inputs); ++i)
+	for (i = 0; i < pipe->num_inputs; ++i) {
+		pipe->inputs[i]->pipe = NULL;
 		pipe->inputs[i] = NULL;
+	}
+
+	pipe->output->pipe = NULL;
+	pipe->output = NULL;
 
 	INIT_LIST_HEAD(&pipe->entities);
 	pipe->state = VSP1_PIPELINE_STOPPED;
 	pipe->buffers_ready = 0;
 	pipe->num_inputs = 0;
-	pipe->output = NULL;
 	pipe->bru = NULL;
 	pipe->lif = NULL;
 	pipe->uds = NULL;
@@ -344,7 +348,7 @@ void vsp1_pipelines_suspend(struct vsp1_device *vsp1)
 		if (wpf == NULL)
 			continue;
 
-		pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
+		pipe = wpf->pipe;
 		if (pipe == NULL)
 			continue;
 
@@ -361,7 +365,7 @@ void vsp1_pipelines_suspend(struct vsp1_device *vsp1)
 		if (wpf == NULL)
 			continue;
 
-		pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
+		pipe = wpf->pipe;
 		if (pipe == NULL)
 			continue;
 
@@ -385,7 +389,7 @@ void vsp1_pipelines_resume(struct vsp1_device *vsp1)
 		if (wpf == NULL)
 			continue;
 
-		pipe = to_vsp1_pipeline(&wpf->entity.subdev.entity);
+		pipe = wpf->pipe;
 		if (pipe == NULL)
 			continue;
 
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h b/drivers/media/platform/vsp1/vsp1_pipe.h
index 1100229a1ed2..9fd688bfe638 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -103,14 +103,6 @@ struct vsp1_pipeline {
 	struct vsp1_dl_list *dl;
 };
 
-static inline struct vsp1_pipeline *to_vsp1_pipeline(struct media_entity *e)
-{
-	if (likely(e->pipe))
-		return container_of(e->pipe, struct vsp1_pipeline, pipe);
-	else
-		return NULL;
-}
-
 void vsp1_pipeline_reset(struct vsp1_pipeline *pipe);
 void vsp1_pipeline_init(struct vsp1_pipeline *pipe);
 
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.h b/drivers/media/platform/vsp1/vsp1_rwpf.h
index 38c8c902db52..9ff7c78f239e 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.h
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.h
@@ -25,6 +25,7 @@
 
 struct v4l2_ctrl;
 struct vsp1_dl_manager;
+struct vsp1_pipeline;
 struct vsp1_rwpf;
 struct vsp1_video;
 
@@ -36,6 +37,7 @@ struct vsp1_rwpf {
 	struct vsp1_entity entity;
 	struct v4l2_ctrl_handler ctrls;
 
+	struct vsp1_pipeline *pipe;
 	struct vsp1_video *video;
 
 	unsigned int max_width;
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index a16a661e5b69..2c642726a259 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -293,10 +293,12 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 			rwpf = to_rwpf(subdev);
 			pipe->inputs[rwpf->entity.index] = rwpf;
 			rwpf->video->pipe_index = ++pipe->num_inputs;
+			rwpf->pipe = pipe;
 		} else if (e->type == VSP1_ENTITY_WPF) {
 			rwpf = to_rwpf(subdev);
 			pipe->output = rwpf;
 			rwpf->video->pipe_index = 0;
+			rwpf->pipe = pipe;
 		} else if (e->type == VSP1_ENTITY_LIF) {
 			pipe->lif = e;
 		} else if (e->type == VSP1_ENTITY_BRU) {
@@ -384,7 +386,7 @@ static void vsp1_video_pipeline_cleanup(struct vsp1_pipeline *pipe)
 static struct vsp1_vb2_buffer *
 vsp1_video_complete_buffer(struct vsp1_video *video)
 {
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
+	struct vsp1_pipeline *pipe = video->rwpf->pipe;
 	struct vsp1_vb2_buffer *next = NULL;
 	struct vsp1_vb2_buffer *done;
 	unsigned long flags;
@@ -563,7 +565,7 @@ static void vsp1_video_buffer_queue(struct vb2_buffer *vb)
 {
 	struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb);
 	struct vsp1_video *video = vb2_get_drv_priv(vb->vb2_queue);
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
+	struct vsp1_pipeline *pipe = video->rwpf->pipe;
 	struct vsp1_vb2_buffer *buf = to_vsp1_vb2_buffer(vbuf);
 	unsigned long flags;
 	bool empty;
@@ -628,7 +630,7 @@ static int vsp1_video_setup_pipeline(struct vsp1_pipeline *pipe)
 static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
 {
 	struct vsp1_video *video = vb2_get_drv_priv(vq);
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
+	struct vsp1_pipeline *pipe = video->rwpf->pipe;
 	unsigned long flags;
 	int ret;
 
@@ -655,7 +657,7 @@ static int vsp1_video_start_streaming(struct vb2_queue *vq, unsigned int count)
 static void vsp1_video_stop_streaming(struct vb2_queue *vq)
 {
 	struct vsp1_video *video = vb2_get_drv_priv(vq);
-	struct vsp1_pipeline *pipe = to_vsp1_pipeline(&video->video.entity);
+	struct vsp1_pipeline *pipe = video->rwpf->pipe;
 	struct vsp1_vb2_buffer *buffer;
 	unsigned long flags;
 	int ret;
@@ -802,8 +804,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	 * FIXME: This is racy, the ioctl is only protected by the video node
 	 * lock.
 	 */
-	pipe = video->video.entity.pipe
-	     ? to_vsp1_pipeline(&video->video.entity) : &video->pipe;
+	pipe = video->rwpf->pipe ? video->rwpf->pipe : &video->pipe;
 
 	ret = media_entity_pipeline_start(&video->video.entity, &pipe->pipe);
 	if (ret < 0)
-- 
2.7.3

[PATCH v2 39/54] v4l: vsp1: video: Reorder functions

[Laurent Pinchart]

Move the pipeline initialization and cleanup functions to prepare for
the next commit. No functional code change is performed here.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_video.c | 266 +++++++++++++++----------------
 1 file changed, 133 insertions(+), 133 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 2c642726a259..4396018d1408 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -172,6 +172,139 @@ static int __vsp1_video_try_format(struct vsp1_video *video,
  * Pipeline Management
  */
 
+/*
+ * vsp1_video_complete_buffer - Complete the current buffer
+ * @video: the video node
+ *
+ * This function completes the current buffer by filling its sequence number,
+ * time stamp and payload size, and hands it back to the videobuf core.
+ *
+ * When operating in DU output mode (deep pipeline to the DU through the LIF),
+ * the VSP1 needs to constantly supply frames to the display. In that case, if
+ * no other buffer is queued, reuse the one that has just been processed instead
+ * of handing it back to the videobuf core.
+ *
+ * Return the next queued buffer or NULL if the queue is empty.
+ */
+static struct vsp1_vb2_buffer *
+vsp1_video_complete_buffer(struct vsp1_video *video)
+{
+	struct vsp1_pipeline *pipe = video->rwpf->pipe;
+	struct vsp1_vb2_buffer *next = NULL;
+	struct vsp1_vb2_buffer *done;
+	unsigned long flags;
+	unsigned int i;
+
+	spin_lock_irqsave(&video->irqlock, flags);
+
+	if (list_empty(&video->irqqueue)) {
+		spin_unlock_irqrestore(&video->irqlock, flags);
+		return NULL;
+	}
+
+	done = list_first_entry(&video->irqqueue,
+				struct vsp1_vb2_buffer, queue);
+
+	/* In DU output mode reuse the buffer if the list is singular. */
+	if (pipe->lif && list_is_singular(&video->irqqueue)) {
+		spin_unlock_irqrestore(&video->irqlock, flags);
+		return done;
+	}
+
+	list_del(&done->queue);
+
+	if (!list_empty(&video->irqqueue))
+		next = list_first_entry(&video->irqqueue,
+					struct vsp1_vb2_buffer, queue);
+
+	spin_unlock_irqrestore(&video->irqlock, flags);
+
+	done->buf.sequence = video->sequence++;
+	done->buf.vb2_buf.timestamp = ktime_get_ns();
+	for (i = 0; i < done->buf.vb2_buf.num_planes; ++i)
+		vb2_set_plane_payload(&done->buf.vb2_buf, i,
+				      vb2_plane_size(&done->buf.vb2_buf, i));
+	vb2_buffer_done(&done->buf.vb2_buf, VB2_BUF_STATE_DONE);
+
+	return next;
+}
+
+static void vsp1_video_frame_end(struct vsp1_pipeline *pipe,
+				 struct vsp1_rwpf *rwpf)
+{
+	struct vsp1_video *video = rwpf->video;
+	struct vsp1_vb2_buffer *buf;
+	unsigned long flags;
+
+	buf = vsp1_video_complete_buffer(video);
+	if (buf == NULL)
+		return;
+
+	spin_lock_irqsave(&pipe->irqlock, flags);
+
+	video->rwpf->mem = buf->mem;
+	pipe->buffers_ready |= 1 << video->pipe_index;
+
+	spin_unlock_irqrestore(&pipe->irqlock, flags);
+}
+
+static void vsp1_video_pipeline_run(struct vsp1_pipeline *pipe)
+{
+	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+	unsigned int i;
+
+	if (!pipe->dl)
+		pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
+
+	for (i = 0; i < vsp1->info->rpf_count; ++i) {
+		struct vsp1_rwpf *rwpf = pipe->inputs[i];
+
+		if (rwpf)
+			vsp1_rwpf_set_memory(rwpf, pipe->dl);
+	}
+
+	if (!pipe->lif)
+		vsp1_rwpf_set_memory(pipe->output, pipe->dl);
+
+	vsp1_dl_list_commit(pipe->dl);
+	pipe->dl = NULL;
+
+	vsp1_pipeline_run(pipe);
+}
+
+static void vsp1_video_pipeline_frame_end(struct vsp1_pipeline *pipe)
+{
+	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
+	enum vsp1_pipeline_state state;
+	unsigned long flags;
+	unsigned int i;
+
+	/* Complete buffers on all video nodes. */
+	for (i = 0; i < vsp1->info->rpf_count; ++i) {
+		if (!pipe->inputs[i])
+			continue;
+
+		vsp1_video_frame_end(pipe, pipe->inputs[i]);
+	}
+
+	vsp1_video_frame_end(pipe, pipe->output);
+
+	spin_lock_irqsave(&pipe->irqlock, flags);
+
+	state = pipe->state;
+	pipe->state = VSP1_PIPELINE_STOPPED;
+
+	/* If a stop has been requested, mark the pipeline as stopped and
+	 * return. Otherwise restart the pipeline if ready.
+	 */
+	if (state == VSP1_PIPELINE_STOPPING)
+		wake_up(&pipe->wq);
+	else if (vsp1_pipeline_ready(pipe))
+		vsp1_video_pipeline_run(pipe);
+
+	spin_unlock_irqrestore(&pipe->irqlock, flags);
+}
+
 static int vsp1_video_pipeline_build_branch(struct vsp1_pipeline *pipe,
 					    struct vsp1_rwpf *input,
 					    struct vsp1_rwpf *output)
@@ -369,139 +502,6 @@ static void vsp1_video_pipeline_cleanup(struct vsp1_pipeline *pipe)
 	mutex_unlock(&pipe->lock);
 }
 
-/*
- * vsp1_video_complete_buffer - Complete the current buffer
- * @video: the video node
- *
- * This function completes the current buffer by filling its sequence number,
- * time stamp and payload size, and hands it back to the videobuf core.
- *
- * When operating in DU output mode (deep pipeline to the DU through the LIF),
- * the VSP1 needs to constantly supply frames to the display. In that case, if
- * no other buffer is queued, reuse the one that has just been processed instead
- * of handing it back to the videobuf core.
- *
- * Return the next queued buffer or NULL if the queue is empty.
- */
-static struct vsp1_vb2_buffer *
-vsp1_video_complete_buffer(struct vsp1_video *video)
-{
-	struct vsp1_pipeline *pipe = video->rwpf->pipe;
-	struct vsp1_vb2_buffer *next = NULL;
-	struct vsp1_vb2_buffer *done;
-	unsigned long flags;
-	unsigned int i;
-
-	spin_lock_irqsave(&video->irqlock, flags);
-
-	if (list_empty(&video->irqqueue)) {
-		spin_unlock_irqrestore(&video->irqlock, flags);
-		return NULL;
-	}
-
-	done = list_first_entry(&video->irqqueue,
-				struct vsp1_vb2_buffer, queue);
-
-	/* In DU output mode reuse the buffer if the list is singular. */
-	if (pipe->lif && list_is_singular(&video->irqqueue)) {
-		spin_unlock_irqrestore(&video->irqlock, flags);
-		return done;
-	}
-
-	list_del(&done->queue);
-
-	if (!list_empty(&video->irqqueue))
-		next = list_first_entry(&video->irqqueue,
-					struct vsp1_vb2_buffer, queue);
-
-	spin_unlock_irqrestore(&video->irqlock, flags);
-
-	done->buf.sequence = video->sequence++;
-	done->buf.vb2_buf.timestamp = ktime_get_ns();
-	for (i = 0; i < done->buf.vb2_buf.num_planes; ++i)
-		vb2_set_plane_payload(&done->buf.vb2_buf, i,
-				      vb2_plane_size(&done->buf.vb2_buf, i));
-	vb2_buffer_done(&done->buf.vb2_buf, VB2_BUF_STATE_DONE);
-
-	return next;
-}
-
-static void vsp1_video_frame_end(struct vsp1_pipeline *pipe,
-				 struct vsp1_rwpf *rwpf)
-{
-	struct vsp1_video *video = rwpf->video;
-	struct vsp1_vb2_buffer *buf;
-	unsigned long flags;
-
-	buf = vsp1_video_complete_buffer(video);
-	if (buf == NULL)
-		return;
-
-	spin_lock_irqsave(&pipe->irqlock, flags);
-
-	video->rwpf->mem = buf->mem;
-	pipe->buffers_ready |= 1 << video->pipe_index;
-
-	spin_unlock_irqrestore(&pipe->irqlock, flags);
-}
-
-static void vsp1_video_pipeline_run(struct vsp1_pipeline *pipe)
-{
-	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
-	unsigned int i;
-
-	if (!pipe->dl)
-		pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
-
-	for (i = 0; i < vsp1->info->rpf_count; ++i) {
-		struct vsp1_rwpf *rwpf = pipe->inputs[i];
-
-		if (rwpf)
-			vsp1_rwpf_set_memory(rwpf, pipe->dl);
-	}
-
-	if (!pipe->lif)
-		vsp1_rwpf_set_memory(pipe->output, pipe->dl);
-
-	vsp1_dl_list_commit(pipe->dl);
-	pipe->dl = NULL;
-
-	vsp1_pipeline_run(pipe);
-}
-
-static void vsp1_video_pipeline_frame_end(struct vsp1_pipeline *pipe)
-{
-	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
-	enum vsp1_pipeline_state state;
-	unsigned long flags;
-	unsigned int i;
-
-	/* Complete buffers on all video nodes. */
-	for (i = 0; i < vsp1->info->rpf_count; ++i) {
-		if (!pipe->inputs[i])
-			continue;
-
-		vsp1_video_frame_end(pipe, pipe->inputs[i]);
-	}
-
-	vsp1_video_frame_end(pipe, pipe->output);
-
-	spin_lock_irqsave(&pipe->irqlock, flags);
-
-	state = pipe->state;
-	pipe->state = VSP1_PIPELINE_STOPPED;
-
-	/* If a stop has been requested, mark the pipeline as stopped and
-	 * return. Otherwise restart the pipeline if ready.
-	 */
-	if (state == VSP1_PIPELINE_STOPPING)
-		wake_up(&pipe->wq);
-	else if (vsp1_pipeline_ready(pipe))
-		vsp1_video_pipeline_run(pipe);
-
-	spin_unlock_irqrestore(&pipe->irqlock, flags);
-}
-
 /* -----------------------------------------------------------------------------
  * videobuf2 Queue Operations
  */
-- 
2.7.3

[PATCH v2 40/54] v4l: vsp1: Allocate pipelines on demand

[Laurent Pinchart]

Instead of embedding pipelines in the vsp1_video objects allocate them
on demand when they are needed. This fixes the streamon race condition
where pipelines objects from different video nodes could be used for the
same pipeline.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c   |   1 +
 drivers/media/platform/vsp1/vsp1_drv.c   |   1 +
 drivers/media/platform/vsp1/vsp1_pipe.c  |   1 +
 drivers/media/platform/vsp1/vsp1_pipe.h  |   5 +-
 drivers/media/platform/vsp1/vsp1_rpf.c   |   1 +
 drivers/media/platform/vsp1/vsp1_video.c | 124 +++++++++++++++++--------------
 drivers/media/platform/vsp1/vsp1_video.h |   2 -
 drivers/media/platform/vsp1/vsp1_wpf.c   |   1 +
 8 files changed, 75 insertions(+), 61 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 820c3c90a4a6..972526b5239e 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -19,6 +19,7 @@
 #include "vsp1.h"
 #include "vsp1_bru.h"
 #include "vsp1_dl.h"
+#include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index f1be2680013d..596f26d81494 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -30,6 +30,7 @@
 #include "vsp1_hsit.h"
 #include "vsp1_lif.h"
 #include "vsp1_lut.h"
+#include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_sru.h"
 #include "vsp1_uds.h"
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 8ac080f87b08..4913b933562c 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -194,6 +194,7 @@ void vsp1_pipeline_init(struct vsp1_pipeline *pipe)
 	mutex_init(&pipe->lock);
 	spin_lock_init(&pipe->irqlock);
 	init_waitqueue_head(&pipe->wq);
+	kref_init(&pipe->kref);
 
 	INIT_LIST_HEAD(&pipe->entities);
 	pipe->state = VSP1_PIPELINE_STOPPED;
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.h b/drivers/media/platform/vsp1/vsp1_pipe.h
index 9fd688bfe638..7b56113511dd 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.h
+++ b/drivers/media/platform/vsp1/vsp1_pipe.h
@@ -13,6 +13,7 @@
 #ifndef __VSP1_PIPE_H__
 #define __VSP1_PIPE_H__
 
+#include <linux/kref.h>
 #include <linux/list.h>
 #include <linux/spinlock.h>
 #include <linux/wait.h>
@@ -63,7 +64,7 @@ enum vsp1_pipeline_state {
  * @wq: work queue to wait for state change completion
  * @frame_end: frame end interrupt handler
  * @lock: protects the pipeline use count and stream count
- * @use_count: number of video nodes using the pipeline
+ * @kref: pipeline reference count
  * @stream_count: number of streaming video nodes
  * @buffers_ready: bitmask of RPFs and WPFs with at least one buffer available
  * @num_inputs: number of RPFs
@@ -86,7 +87,7 @@ struct vsp1_pipeline {
 	void (*frame_end)(struct vsp1_pipeline *pipe);
 
 	struct mutex lock;
-	unsigned int use_count;
+	struct kref kref;
 	unsigned int stream_count;
 	unsigned int buffers_ready;
 
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index e373f1473c1f..bc94427c0740 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -17,6 +17,7 @@
 
 #include "vsp1.h"
 #include "vsp1_dl.h"
+#include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
diff --git a/drivers/media/platform/vsp1/vsp1_video.c b/drivers/media/platform/vsp1/vsp1_video.c
index 4396018d1408..a9aec5c0bec6 100644
--- a/drivers/media/platform/vsp1/vsp1_video.c
+++ b/drivers/media/platform/vsp1/vsp1_video.c
@@ -399,14 +399,10 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 	unsigned int i;
 	int ret;
 
-	mutex_lock(&mdev->graph_mutex);
-
 	/* Walk the graph to locate the entities and video nodes. */
 	ret = media_entity_graph_walk_init(&graph, mdev);
-	if (ret) {
-		mutex_unlock(&mdev->graph_mutex);
+	if (ret)
 		return ret;
-	}
 
 	media_entity_graph_walk_start(&graph, entity);
 
@@ -439,15 +435,11 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 		}
 	}
 
-	mutex_unlock(&mdev->graph_mutex);
-
 	media_entity_graph_walk_cleanup(&graph);
 
 	/* We need one output and at least one input. */
-	if (pipe->num_inputs == 0 || !pipe->output) {
-		ret = -EPIPE;
-		goto error;
-	}
+	if (pipe->num_inputs == 0 || !pipe->output)
+		return -EPIPE;
 
 	/* Follow links downstream for each input and make sure the graph
 	 * contains no loop and that all branches end at the output WPF.
@@ -459,47 +451,66 @@ static int vsp1_video_pipeline_build(struct vsp1_pipeline *pipe,
 		ret = vsp1_video_pipeline_build_branch(pipe, pipe->inputs[i],
 						       pipe->output);
 		if (ret < 0)
-			goto error;
+			return ret;
 	}
 
 	return 0;
-
-error:
-	vsp1_pipeline_reset(pipe);
-	return ret;
 }
 
 static int vsp1_video_pipeline_init(struct vsp1_pipeline *pipe,
 				    struct vsp1_video *video)
 {
+	vsp1_pipeline_init(pipe);
+
+	pipe->frame_end = vsp1_video_pipeline_frame_end;
+
+	return vsp1_video_pipeline_build(pipe, video);
+}
+
+static struct vsp1_pipeline *vsp1_video_pipeline_get(struct vsp1_video *video)
+{
+	struct vsp1_pipeline *pipe;
 	int ret;
 
-	mutex_lock(&pipe->lock);
+	/* Get a pipeline object for the video node. If a pipeline has already
+	 * been allocated just increment its reference count and return it.
+	 * Otherwise allocate a new pipeline and initialize it, it will be freed
+	 * when the last reference is released.
+	 */
+	if (!video->rwpf->pipe) {
+		pipe = kzalloc(sizeof(*pipe), GFP_KERNEL);
+		if (!pipe)
+			return ERR_PTR(-ENOMEM);
 
-	/* If we're the first user build and validate the pipeline. */
-	if (pipe->use_count == 0) {
-		ret = vsp1_video_pipeline_build(pipe, video);
-		if (ret < 0)
-			goto done;
+		ret = vsp1_video_pipeline_init(pipe, video);
+		if (ret < 0) {
+			vsp1_pipeline_reset(pipe);
+			kfree(pipe);
+			return ERR_PTR(ret);
+		}
+	} else {
+		pipe = video->rwpf->pipe;
+		kref_get(&pipe->kref);
 	}
 
-	pipe->use_count++;
-	ret = 0;
-
-done:
-	mutex_unlock(&pipe->lock);
-	return ret;
+	return pipe;
 }
 
-static void vsp1_video_pipeline_cleanup(struct vsp1_pipeline *pipe)
+static void vsp1_video_pipeline_release(struct kref *kref)
 {
-	mutex_lock(&pipe->lock);
+	struct vsp1_pipeline *pipe = container_of(kref, typeof(*pipe), kref);
 
-	/* If we're the last user clean up the pipeline. */
-	if (--pipe->use_count == 0)
-		vsp1_pipeline_reset(pipe);
+	vsp1_pipeline_reset(pipe);
+	kfree(pipe);
+}
 
-	mutex_unlock(&pipe->lock);
+static void vsp1_video_pipeline_put(struct vsp1_pipeline *pipe)
+{
+	struct media_device *mdev = &pipe->output->entity.vsp1->media_dev;
+
+	mutex_lock(&mdev->graph_mutex);
+	kref_put(&pipe->kref, vsp1_video_pipeline_release);
+	mutex_unlock(&mdev->graph_mutex);
 }
 
 /* -----------------------------------------------------------------------------
@@ -674,8 +685,8 @@ static void vsp1_video_stop_streaming(struct vb2_queue *vq)
 	}
 	mutex_unlock(&pipe->lock);
 
-	vsp1_video_pipeline_cleanup(pipe);
 	media_entity_pipeline_stop(&video->video.entity);
+	vsp1_video_pipeline_put(pipe);
 
 	/* Remove all buffers from the IRQ queue. */
 	spin_lock_irqsave(&video->irqlock, flags);
@@ -787,6 +798,7 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 {
 	struct v4l2_fh *vfh = file->private_data;
 	struct vsp1_video *video = to_vsp1_video(vfh->vdev);
+	struct media_device *mdev = &video->vsp1->media_dev;
 	struct vsp1_pipeline *pipe;
 	int ret;
 
@@ -795,20 +807,25 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 
 	video->sequence = 0;
 
-	/* Start streaming on the pipeline. No link touching an entity in the
-	 * pipeline can be activated or deactivated once streaming is started.
-	 *
-	 * Use the VSP1 pipeline object embedded in the first video object that
-	 * starts streaming.
-	 *
-	 * FIXME: This is racy, the ioctl is only protected by the video node
-	 * lock.
+	/* Get a pipeline for the video node and start streaming on it. No link
+	 * touching an entity in the pipeline can be activated or deactivated
+	 * once streaming is started.
 	 */
-	pipe = video->rwpf->pipe ? video->rwpf->pipe : &video->pipe;
+	mutex_lock(&mdev->graph_mutex);
 
-	ret = media_entity_pipeline_start(&video->video.entity, &pipe->pipe);
-	if (ret < 0)
-		return ret;
+	pipe = vsp1_video_pipeline_get(video);
+	if (IS_ERR(pipe)) {
+		mutex_unlock(&mdev->graph_mutex);
+		return PTR_ERR(pipe);
+	}
+
+	ret = __media_entity_pipeline_start(&video->video.entity, &pipe->pipe);
+	if (ret < 0) {
+		mutex_unlock(&mdev->graph_mutex);
+		goto err_pipe;
+	}
+
+	mutex_unlock(&mdev->graph_mutex);
 
 	/* Verify that the configured format matches the output of the connected
 	 * subdev.
@@ -817,21 +834,17 @@ vsp1_video_streamon(struct file *file, void *fh, enum v4l2_buf_type type)
 	if (ret < 0)
 		goto err_stop;
 
-	ret = vsp1_video_pipeline_init(pipe, video);
-	if (ret < 0)
-		goto err_stop;
-
 	/* Start the queue. */
 	ret = vb2_streamon(&video->queue, type);
 	if (ret < 0)
-		goto err_cleanup;
+		goto err_stop;
 
 	return 0;
 
-err_cleanup:
-	vsp1_video_pipeline_cleanup(pipe);
 err_stop:
 	media_entity_pipeline_stop(&video->video.entity);
+err_pipe:
+	vsp1_video_pipeline_put(pipe);
 	return ret;
 }
 
@@ -947,9 +960,6 @@ struct vsp1_video *vsp1_video_create(struct vsp1_device *vsp1,
 	spin_lock_init(&video->irqlock);
 	INIT_LIST_HEAD(&video->irqqueue);
 
-	vsp1_pipeline_init(&video->pipe);
-	video->pipe.frame_end = vsp1_video_pipeline_frame_end;
-
 	/* Initialize the media entity... */
 	ret = media_entity_pads_init(&video->video.entity, 1, &video->pad);
 	if (ret < 0)
diff --git a/drivers/media/platform/vsp1/vsp1_video.h b/drivers/media/platform/vsp1/vsp1_video.h
index 64abd39ee1e7..867b00807c46 100644
--- a/drivers/media/platform/vsp1/vsp1_video.h
+++ b/drivers/media/platform/vsp1/vsp1_video.h
@@ -18,7 +18,6 @@
 
 #include <media/videobuf2-v4l2.h>
 
-#include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
 
 struct vsp1_vb2_buffer {
@@ -44,7 +43,6 @@ struct vsp1_video {
 
 	struct mutex lock;
 
-	struct vsp1_pipeline pipe;
 	unsigned int pipe_index;
 
 	struct vb2_queue queue;
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index 5aa5eb4b8121..a88ed0fc69ac 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -17,6 +17,7 @@
 
 #include "vsp1.h"
 #include "vsp1_dl.h"
+#include "vsp1_pipe.h"
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
-- 
2.7.3

[PATCH v2 41/54] v4l: vsp1: RPF entities can't be target nodes

[Laurent Pinchart]

The RPF entities are located at the very beginning of pipelines, they
can't be target nodes in the Data Path Router matrix. Remove their input
ID from the routing table.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_entity.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 36d425234cd3..e4f832c764a6 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -160,11 +160,11 @@ static const struct vsp1_route vsp1_routes[] = {
 	{ VSP1_ENTITY_HST, 0, VI6_DPR_HST_ROUTE, { VI6_DPR_NODE_HST, } },
 	{ VSP1_ENTITY_LIF, 0, 0, { VI6_DPR_NODE_LIF, } },
 	{ VSP1_ENTITY_LUT, 0, VI6_DPR_LUT_ROUTE, { VI6_DPR_NODE_LUT, } },
-	{ VSP1_ENTITY_RPF, 0, VI6_DPR_RPF_ROUTE(0), { VI6_DPR_NODE_RPF(0), } },
-	{ VSP1_ENTITY_RPF, 1, VI6_DPR_RPF_ROUTE(1), { VI6_DPR_NODE_RPF(1), } },
-	{ VSP1_ENTITY_RPF, 2, VI6_DPR_RPF_ROUTE(2), { VI6_DPR_NODE_RPF(2), } },
-	{ VSP1_ENTITY_RPF, 3, VI6_DPR_RPF_ROUTE(3), { VI6_DPR_NODE_RPF(3), } },
-	{ VSP1_ENTITY_RPF, 4, VI6_DPR_RPF_ROUTE(4), { VI6_DPR_NODE_RPF(4), } },
+	{ VSP1_ENTITY_RPF, 0, VI6_DPR_RPF_ROUTE(0), { 0, } },
+	{ VSP1_ENTITY_RPF, 1, VI6_DPR_RPF_ROUTE(1), { 0, } },
+	{ VSP1_ENTITY_RPF, 2, VI6_DPR_RPF_ROUTE(2), { 0, } },
+	{ VSP1_ENTITY_RPF, 3, VI6_DPR_RPF_ROUTE(3), { 0, } },
+	{ VSP1_ENTITY_RPF, 4, VI6_DPR_RPF_ROUTE(4), { 0, } },
 	{ VSP1_ENTITY_SRU, 0, VI6_DPR_SRU_ROUTE, { VI6_DPR_NODE_SRU, } },
 	{ VSP1_ENTITY_UDS, 0, VI6_DPR_UDS_ROUTE(0), { VI6_DPR_NODE_UDS(0), } },
 	{ VSP1_ENTITY_UDS, 1, VI6_DPR_UDS_ROUTE(1), { VI6_DPR_NODE_UDS(1), } },
-- 
2.7.3

[PATCH v2 42/54] v4l: vsp1: Factorize get pad format code

[Laurent Pinchart]

All entities implement the same get pad format handler, factorize it
into a common function.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 19 +------------------
 drivers/media/platform/vsp1/vsp1_entity.c | 25 +++++++++++++++++++++++++
 drivers/media/platform/vsp1/vsp1_entity.h |  4 ++++
 drivers/media/platform/vsp1/vsp1_hsit.c   | 19 +------------------
 drivers/media/platform/vsp1/vsp1_lif.c    | 19 +------------------
 drivers/media/platform/vsp1/vsp1_lut.c    | 19 +------------------
 drivers/media/platform/vsp1/vsp1_rwpf.c   | 19 +------------------
 drivers/media/platform/vsp1/vsp1_sru.c    | 19 +------------------
 drivers/media/platform/vsp1/vsp1_uds.c    | 19 +------------------
 9 files changed, 36 insertions(+), 126 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index 972526b5239e..fa293ee2829d 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -129,23 +129,6 @@ static struct v4l2_rect *bru_get_compose(struct vsp1_bru *bru,
 	return v4l2_subdev_get_try_compose(&bru->entity.subdev, cfg, pad);
 }
 
-static int bru_get_format(struct v4l2_subdev *subdev,
-			  struct v4l2_subdev_pad_config *cfg,
-			  struct v4l2_subdev_format *fmt)
-{
-	struct vsp1_bru *bru = to_bru(subdev);
-	struct v4l2_subdev_pad_config *config;
-
-	config = vsp1_entity_get_pad_config(&bru->entity, cfg, fmt->which);
-	if (!config)
-		return -EINVAL;
-
-	fmt->format = *vsp1_entity_get_pad_format(&bru->entity, config,
-						  fmt->pad);
-
-	return 0;
-}
-
 static void bru_try_format(struct vsp1_bru *bru,
 			   struct v4l2_subdev_pad_config *config,
 			   unsigned int pad, struct v4l2_mbus_framefmt *fmt)
@@ -292,7 +275,7 @@ static struct v4l2_subdev_pad_ops bru_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = bru_enum_mbus_code,
 	.enum_frame_size = bru_enum_frame_size,
-	.get_fmt = bru_get_format,
+	.get_fmt = vsp1_subdev_get_pad_format,
 	.set_fmt = bru_set_format,
 	.get_selection = bru_get_selection,
 	.set_selection = bru_set_selection,
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index e4f832c764a6..5030974f3083 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -116,6 +116,31 @@ int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+/*
+ * vsp1_subdev_get_pad_format - Subdev pad get_fmt handler
+ * @subdev: V4L2 subdevice
+ * @cfg: V4L2 subdev pad configuration
+ * @fmt: V4L2 subdev format
+ *
+ * This function implements the subdev get_fmt pad operation. It can be used as
+ * a direct drop-in for the operation handler.
+ */
+int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_pad_config *cfg,
+			       struct v4l2_subdev_format *fmt)
+{
+	struct vsp1_entity *entity = to_vsp1_entity(subdev);
+	struct v4l2_subdev_pad_config *config;
+
+	config = vsp1_entity_get_pad_config(entity, cfg, fmt->which);
+	if (!config)
+		return -EINVAL;
+
+	fmt->format = *vsp1_entity_get_pad_format(entity, config, fmt->pad);
+
+	return 0;
+}
+
 /* -----------------------------------------------------------------------------
  * Media Operations
  */
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index b58bcb15c087..7845e931138f 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -127,4 +127,8 @@ int vsp1_entity_init_cfg(struct v4l2_subdev *subdev,
 void vsp1_entity_route_setup(struct vsp1_entity *source,
 			     struct vsp1_dl_list *dl);
 
+int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_pad_config *cfg,
+			       struct v4l2_subdev_format *fmt);
+
 #endif /* __VSP1_ENTITY_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index 1b67ba9a69ed..af810eb1fdb0 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -90,23 +90,6 @@ static int hsit_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int hsit_get_format(struct v4l2_subdev *subdev,
-			   struct v4l2_subdev_pad_config *cfg,
-			   struct v4l2_subdev_format *fmt)
-{
-	struct vsp1_hsit *hsit = to_hsit(subdev);
-	struct v4l2_subdev_pad_config *config;
-
-	config = vsp1_entity_get_pad_config(&hsit->entity, cfg, fmt->which);
-	if (!config)
-		return -EINVAL;
-
-	fmt->format = *vsp1_entity_get_pad_format(&hsit->entity, config,
-						  fmt->pad);
-
-	return 0;
-}
-
 static int hsit_set_format(struct v4l2_subdev *subdev,
 			   struct v4l2_subdev_pad_config *cfg,
 			   struct v4l2_subdev_format *fmt)
@@ -154,7 +137,7 @@ static struct v4l2_subdev_pad_ops hsit_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = hsit_enum_mbus_code,
 	.enum_frame_size = hsit_enum_frame_size,
-	.get_fmt = hsit_get_format,
+	.get_fmt = vsp1_subdev_get_pad_format,
 	.set_fmt = hsit_set_format,
 };
 
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index 628109aba23b..db698e7d4884 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -107,23 +107,6 @@ static int lif_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int lif_get_format(struct v4l2_subdev *subdev,
-			  struct v4l2_subdev_pad_config *cfg,
-			  struct v4l2_subdev_format *fmt)
-{
-	struct vsp1_lif *lif = to_lif(subdev);
-	struct v4l2_subdev_pad_config *config;
-
-	config = vsp1_entity_get_pad_config(&lif->entity, cfg, fmt->which);
-	if (!config)
-		return -EINVAL;
-
-	fmt->format = *vsp1_entity_get_pad_format(&lif->entity, config,
-						  fmt->pad);
-
-	return 0;
-}
-
 static int lif_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
@@ -173,7 +156,7 @@ static struct v4l2_subdev_pad_ops lif_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = lif_enum_mbus_code,
 	.enum_frame_size = lif_enum_frame_size,
-	.get_fmt = lif_get_format,
+	.get_fmt = vsp1_subdev_get_pad_format,
 	.set_fmt = lif_set_format,
 };
 
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index fba6cc4e67a1..6a0f10f71dda 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -136,23 +136,6 @@ static int lut_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int lut_get_format(struct v4l2_subdev *subdev,
-			  struct v4l2_subdev_pad_config *cfg,
-			  struct v4l2_subdev_format *fmt)
-{
-	struct vsp1_lut *lut = to_lut(subdev);
-	struct v4l2_subdev_pad_config *config;
-
-	config = vsp1_entity_get_pad_config(&lut->entity, cfg, fmt->which);
-	if (!config)
-		return -EINVAL;
-
-	fmt->format = *vsp1_entity_get_pad_format(&lut->entity, config,
-						  fmt->pad);
-
-	return 0;
-}
-
 static int lut_set_format(struct v4l2_subdev *subdev,
 			  struct v4l2_subdev_pad_config *cfg,
 			  struct v4l2_subdev_format *fmt)
@@ -208,7 +191,7 @@ static struct v4l2_subdev_pad_ops lut_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = lut_enum_mbus_code,
 	.enum_frame_size = lut_enum_frame_size,
-	.get_fmt = lut_get_format,
+	.get_fmt = vsp1_subdev_get_pad_format,
 	.set_fmt = lut_set_format,
 };
 
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 4d302f5cccb2..64d649a1bcf5 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -83,23 +83,6 @@ static int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int vsp1_rwpf_get_format(struct v4l2_subdev *subdev,
-				struct v4l2_subdev_pad_config *cfg,
-				struct v4l2_subdev_format *fmt)
-{
-	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
-	struct v4l2_subdev_pad_config *config;
-
-	config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, fmt->which);
-	if (!config)
-		return -EINVAL;
-
-	fmt->format = *vsp1_entity_get_pad_format(&rwpf->entity, config,
-						  fmt->pad);
-
-	return 0;
-}
-
 static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
 				struct v4l2_subdev_pad_config *cfg,
 				struct v4l2_subdev_format *fmt)
@@ -254,7 +237,7 @@ const struct v4l2_subdev_pad_ops vsp1_rwpf_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = vsp1_rwpf_enum_mbus_code,
 	.enum_frame_size = vsp1_rwpf_enum_frame_size,
-	.get_fmt = vsp1_rwpf_get_format,
+	.get_fmt = vsp1_subdev_get_pad_format,
 	.set_fmt = vsp1_rwpf_set_format,
 	.get_selection = vsp1_rwpf_get_selection,
 	.set_selection = vsp1_rwpf_set_selection,
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 3dc335d1499d..38fb4e1e8bae 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -184,23 +184,6 @@ static int sru_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int sru_get_format(struct v4l2_subdev *subdev,
-			  struct v4l2_subdev_pad_config *cfg,
-			  struct v4l2_subdev_format *fmt)
-{
-	struct vsp1_sru *sru = to_sru(subdev);
-	struct v4l2_subdev_pad_config *config;
-
-	config = vsp1_entity_get_pad_config(&sru->entity, cfg, fmt->which);
-	if (!config)
-		return -EINVAL;
-
-	fmt->format = *vsp1_entity_get_pad_format(&sru->entity, config,
-						  fmt->pad);
-
-	return 0;
-}
-
 static void sru_try_format(struct vsp1_sru *sru,
 			   struct v4l2_subdev_pad_config *config,
 			   unsigned int pad, struct v4l2_mbus_framefmt *fmt)
@@ -285,7 +268,7 @@ static struct v4l2_subdev_pad_ops sru_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = sru_enum_mbus_code,
 	.enum_frame_size = sru_enum_frame_size,
-	.get_fmt = sru_get_format,
+	.get_fmt = vsp1_subdev_get_pad_format,
 	.set_fmt = sru_set_format,
 };
 
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 0bc0616f6b18..59432c7c29b9 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -182,23 +182,6 @@ static int uds_enum_frame_size(struct v4l2_subdev *subdev,
 	return 0;
 }
 
-static int uds_get_format(struct v4l2_subdev *subdev,
-			  struct v4l2_subdev_pad_config *cfg,
-			  struct v4l2_subdev_format *fmt)
-{
-	struct vsp1_uds *uds = to_uds(subdev);
-	struct v4l2_subdev_pad_config *config;
-
-	config = vsp1_entity_get_pad_config(&uds->entity, cfg, fmt->which);
-	if (!config)
-		return -EINVAL;
-
-	fmt->format = *vsp1_entity_get_pad_format(&uds->entity, config,
-						  fmt->pad);
-
-	return 0;
-}
-
 static void uds_try_format(struct vsp1_uds *uds,
 			   struct v4l2_subdev_pad_config *config,
 			   unsigned int pad, struct v4l2_mbus_framefmt *fmt)
@@ -272,7 +255,7 @@ static struct v4l2_subdev_pad_ops uds_pad_ops = {
 	.init_cfg = vsp1_entity_init_cfg,
 	.enum_mbus_code = uds_enum_mbus_code,
 	.enum_frame_size = uds_enum_frame_size,
-	.get_fmt = uds_get_format,
+	.get_fmt = vsp1_subdev_get_pad_format,
 	.set_fmt = uds_set_format,
 };
 
-- 
2.7.3

[PATCH v2 43/54] v4l: vsp1: Factorize media bus codes enumeration code

[Laurent Pinchart]

Most of the entities can't perform format conversion and implement the
same media bus enumeration function. Factorize the code into a single
implementation.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_bru.c    | 26 ++---------------
 drivers/media/platform/vsp1/vsp1_entity.c | 46 +++++++++++++++++++++++++++++++
 drivers/media/platform/vsp1/vsp1_entity.h |  4 +++
 drivers/media/platform/vsp1/vsp1_lif.c    | 29 ++-----------------
 drivers/media/platform/vsp1/vsp1_lut.c    | 29 ++-----------------
 drivers/media/platform/vsp1/vsp1_sru.c    | 29 ++-----------------
 drivers/media/platform/vsp1/vsp1_uds.c    | 29 ++-----------------
 7 files changed, 60 insertions(+), 132 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_bru.c b/drivers/media/platform/vsp1/vsp1_bru.c
index fa293ee2829d..835593dd88b3 100644
--- a/drivers/media/platform/vsp1/vsp1_bru.c
+++ b/drivers/media/platform/vsp1/vsp1_bru.c
@@ -76,31 +76,9 @@ static int bru_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_ARGB8888_1X32,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
-	struct vsp1_bru *bru = to_bru(subdev);
-
-	if (code->pad == BRU_PAD_SINK(0)) {
-		if (code->index >= ARRAY_SIZE(codes))
-			return -EINVAL;
-
-		code->code = codes[code->index];
-	} else {
-		struct v4l2_subdev_pad_config *config;
-		struct v4l2_mbus_framefmt *format;
 
-		if (code->index)
-			return -EINVAL;
-
-		config = vsp1_entity_get_pad_config(&bru->entity, cfg,
-						    code->which);
-		if (!config)
-			return -EINVAL;
-
-		format = vsp1_entity_get_pad_format(&bru->entity, config,
-						    BRU_PAD_SINK(0));
-		code->code = format->code;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+					  ARRAY_SIZE(codes));
 }
 
 static int bru_enum_frame_size(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index 5030974f3083..b347442bc662 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -141,6 +141,52 @@ int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+/*
+ * vsp1_subdev_enum_mbus_code - Subdev pad enum_mbus_code handler
+ * @subdev: V4L2 subdevice
+ * @cfg: V4L2 subdev pad configuration
+ * @code: Media bus code enumeration
+ * @codes: Array of supported media bus codes
+ * @ncodes: Number of supported media bus codes
+ *
+ * This function implements the subdev enum_mbus_code pad operation for entities
+ * that do not support format conversion. It enumerates the given supported
+ * media bus codes on the sink pad and reports a source pad format identical to
+ * the sink pad.
+ */
+int vsp1_subdev_enum_mbus_code(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_pad_config *cfg,
+			       struct v4l2_subdev_mbus_code_enum *code,
+			       const unsigned int *codes, unsigned int ncodes)
+{
+	struct vsp1_entity *entity = to_vsp1_entity(subdev);
+
+	if (code->pad == 0) {
+		if (code->index >= ncodes)
+			return -EINVAL;
+
+		code->code = codes[code->index];
+	} else {
+		struct v4l2_subdev_pad_config *config;
+		struct v4l2_mbus_framefmt *format;
+
+		/* The entity can't perform format conversion, the sink format
+		 * is always identical to the source format.
+		 */
+		if (code->index)
+			return -EINVAL;
+
+		config = vsp1_entity_get_pad_config(entity, cfg, code->which);
+		if (!config)
+			return -EINVAL;
+
+		format = vsp1_entity_get_pad_format(entity, config, 0);
+		code->code = format->code;
+	}
+
+	return 0;
+}
+
 /* -----------------------------------------------------------------------------
  * Media Operations
  */
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index 7845e931138f..fbde42256522 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -130,5 +130,9 @@ void vsp1_entity_route_setup(struct vsp1_entity *source,
 int vsp1_subdev_get_pad_format(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_pad_config *cfg,
 			       struct v4l2_subdev_format *fmt);
+int vsp1_subdev_enum_mbus_code(struct v4l2_subdev *subdev,
+			       struct v4l2_subdev_pad_config *cfg,
+			       struct v4l2_subdev_mbus_code_enum *code,
+			       const unsigned int *codes, unsigned int ncodes);
 
 #endif /* __VSP1_ENTITY_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index db698e7d4884..ded4c5a87d4c 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -45,34 +45,9 @@ static int lif_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_ARGB8888_1X32,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
-	struct vsp1_lif *lif = to_lif(subdev);
-
-	if (code->pad == LIF_PAD_SINK) {
-		if (code->index >= ARRAY_SIZE(codes))
-			return -EINVAL;
-
-		code->code = codes[code->index];
-	} else {
-		struct v4l2_subdev_pad_config *config;
-		struct v4l2_mbus_framefmt *format;
-
-		/* The LIF can't perform format conversion, the sink format is
-		 * always identical to the source format.
-		 */
-		if (code->index)
-			return -EINVAL;
-
-		config = vsp1_entity_get_pad_config(&lif->entity, cfg,
-						    code->which);
-		if (!config)
-			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&lif->entity, config,
-						    LIF_PAD_SINK);
-		code->code = format->code;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+					  ARRAY_SIZE(codes));
 }
 
 static int lif_enum_frame_size(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 6a0f10f71dda..781ea99abccf 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -71,34 +71,9 @@ static int lut_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_AHSV8888_1X32,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
-	struct vsp1_lut *lut = to_lut(subdev);
-
-	if (code->pad == LUT_PAD_SINK) {
-		if (code->index >= ARRAY_SIZE(codes))
-			return -EINVAL;
-
-		code->code = codes[code->index];
-	} else {
-		struct v4l2_subdev_pad_config *config;
-		struct v4l2_mbus_framefmt *format;
-
-		/* The LUT can't perform format conversion, the sink format is
-		 * always identical to the source format.
-		 */
-		if (code->index)
-			return -EINVAL;
-
-		config = vsp1_entity_get_pad_config(&lut->entity, cfg,
-						    code->which);
-		if (!config)
-			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&lut->entity, config,
-						    LUT_PAD_SINK);
-		code->code = format->code;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+					  ARRAY_SIZE(codes));
 }
 
 static int lut_enum_frame_size(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/vsp1/vsp1_sru.c b/drivers/media/platform/vsp1/vsp1_sru.c
index 38fb4e1e8bae..9dc7c77c74f8 100644
--- a/drivers/media/platform/vsp1/vsp1_sru.c
+++ b/drivers/media/platform/vsp1/vsp1_sru.c
@@ -116,34 +116,9 @@ static int sru_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_ARGB8888_1X32,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
-	struct vsp1_sru *sru = to_sru(subdev);
-
-	if (code->pad == SRU_PAD_SINK) {
-		if (code->index >= ARRAY_SIZE(codes))
-			return -EINVAL;
-
-		code->code = codes[code->index];
-	} else {
-		struct v4l2_subdev_pad_config *config;
-		struct v4l2_mbus_framefmt *format;
-
-		/* The SRU can't perform format conversion, the sink format is
-		 * always identical to the source format.
-		 */
-		if (code->index)
-			return -EINVAL;
 
-		config = vsp1_entity_get_pad_config(&sru->entity, cfg,
-						    code->which);
-		if (!config)
-			return -EINVAL;
-
-		format = vsp1_entity_get_pad_format(&sru->entity, config,
-						    SRU_PAD_SINK);
-		code->code = format->code;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+					  ARRAY_SIZE(codes));
 }
 
 static int sru_enum_frame_size(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/vsp1/vsp1_uds.c b/drivers/media/platform/vsp1/vsp1_uds.c
index 59432c7c29b9..26f7393d278e 100644
--- a/drivers/media/platform/vsp1/vsp1_uds.c
+++ b/drivers/media/platform/vsp1/vsp1_uds.c
@@ -119,34 +119,9 @@ static int uds_enum_mbus_code(struct v4l2_subdev *subdev,
 		MEDIA_BUS_FMT_ARGB8888_1X32,
 		MEDIA_BUS_FMT_AYUV8_1X32,
 	};
-	struct vsp1_uds *uds = to_uds(subdev);
-
-	if (code->pad == UDS_PAD_SINK) {
-		if (code->index >= ARRAY_SIZE(codes))
-			return -EINVAL;
-
-		code->code = codes[code->index];
-	} else {
-		struct v4l2_subdev_pad_config *config;
-		struct v4l2_mbus_framefmt *format;
-
-		config = vsp1_entity_get_pad_config(&uds->entity, cfg,
-						    code->which);
-		if (!config)
-			return -EINVAL;
-
-		/* The UDS can't perform format conversion, the sink format is
-		 * always identical to the source format.
-		 */
-		if (code->index)
-			return -EINVAL;
 
-		format = vsp1_entity_get_pad_format(&uds->entity, config,
-						    UDS_PAD_SINK);
-		code->code = format->code;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_mbus_code(subdev, cfg, code, codes,
+					  ARRAY_SIZE(codes));
 }
 
 static int uds_enum_frame_size(struct v4l2_subdev *subdev,
-- 
2.7.3

[PATCH v2 44/54] v4l: vsp1: Factorize frame size enumeration code

[Laurent Pinchart]

Most of the entities can't perform scaling and implement the same frame
size enumeration function. Factorize the code into a single
implementation.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_entity.c | 52 +++++++++++++++++++++++++++++++
 drivers/media/platform/vsp1/vsp1_entity.h |  5 +++
 drivers/media/platform/vsp1/vsp1_hsit.c   | 32 ++-----------------
 drivers/media/platform/vsp1/vsp1_lif.c    | 29 ++---------------
 drivers/media/platform/vsp1/vsp1_lut.c    | 32 ++-----------------
 drivers/media/platform/vsp1/vsp1_rwpf.c   | 30 ++----------------
 6 files changed, 69 insertions(+), 111 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_entity.c b/drivers/media/platform/vsp1/vsp1_entity.c
index b347442bc662..01a2e9958494 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.c
+++ b/drivers/media/platform/vsp1/vsp1_entity.c
@@ -187,6 +187,58 @@ int vsp1_subdev_enum_mbus_code(struct v4l2_subdev *subdev,
 	return 0;
 }
 
+/*
+ * vsp1_subdev_enum_frame_size - Subdev pad enum_frame_size handler
+ * @subdev: V4L2 subdevice
+ * @cfg: V4L2 subdev pad configuration
+ * @fse: Frame size enumeration
+ * @min_width: Minimum image width
+ * @min_height: Minimum image height
+ * @max_width: Maximum image width
+ * @max_height: Maximum image height
+ *
+ * This function implements the subdev enum_frame_size pad operation for
+ * entities that do not support scaling or cropping. It reports the given
+ * minimum and maximum frame width and height on the sink pad, and a fixed
+ * source pad size identical to the sink pad.
+ */
+int vsp1_subdev_enum_frame_size(struct v4l2_subdev *subdev,
+				struct v4l2_subdev_pad_config *cfg,
+				struct v4l2_subdev_frame_size_enum *fse,
+				unsigned int min_width, unsigned int min_height,
+				unsigned int max_width, unsigned int max_height)
+{
+	struct vsp1_entity *entity = to_vsp1_entity(subdev);
+	struct v4l2_subdev_pad_config *config;
+	struct v4l2_mbus_framefmt *format;
+
+	config = vsp1_entity_get_pad_config(entity, cfg, fse->which);
+	if (!config)
+		return -EINVAL;
+
+	format = vsp1_entity_get_pad_format(entity, config, fse->pad);
+
+	if (fse->index || fse->code != format->code)
+		return -EINVAL;
+
+	if (fse->pad == 0) {
+		fse->min_width = min_width;
+		fse->max_width = max_width;
+		fse->min_height = min_height;
+		fse->max_height = max_height;
+	} else {
+		/* The size on the source pad are fixed and always identical to
+		 * the size on the sink pad.
+		 */
+		fse->min_width = format->width;
+		fse->max_width = format->width;
+		fse->min_height = format->height;
+		fse->max_height = format->height;
+	}
+
+	return 0;
+}
+
 /* -----------------------------------------------------------------------------
  * Media Operations
  */
diff --git a/drivers/media/platform/vsp1/vsp1_entity.h b/drivers/media/platform/vsp1/vsp1_entity.h
index fbde42256522..130a919052d4 100644
--- a/drivers/media/platform/vsp1/vsp1_entity.h
+++ b/drivers/media/platform/vsp1/vsp1_entity.h
@@ -134,5 +134,10 @@ int vsp1_subdev_enum_mbus_code(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_pad_config *cfg,
 			       struct v4l2_subdev_mbus_code_enum *code,
 			       const unsigned int *codes, unsigned int ncodes);
+int vsp1_subdev_enum_frame_size(struct v4l2_subdev *subdev,
+				struct v4l2_subdev_pad_config *cfg,
+				struct v4l2_subdev_frame_size_enum *fse,
+				unsigned int min_w, unsigned int min_h,
+				unsigned int max_w, unsigned int max_h);
 
 #endif /* __VSP1_ENTITY_H__ */
diff --git a/drivers/media/platform/vsp1/vsp1_hsit.c b/drivers/media/platform/vsp1/vsp1_hsit.c
index af810eb1fdb0..da0b2d9ac39d 100644
--- a/drivers/media/platform/vsp1/vsp1_hsit.c
+++ b/drivers/media/platform/vsp1/vsp1_hsit.c
@@ -59,35 +59,9 @@ static int hsit_enum_frame_size(struct v4l2_subdev *subdev,
 				struct v4l2_subdev_pad_config *cfg,
 				struct v4l2_subdev_frame_size_enum *fse)
 {
-	struct vsp1_hsit *hsit = to_hsit(subdev);
-	struct v4l2_subdev_pad_config *config;
-	struct v4l2_mbus_framefmt *format;
-
-	config = vsp1_entity_get_pad_config(&hsit->entity, cfg, fse->which);
-	if (!config)
-		return -EINVAL;
-
-	format = vsp1_entity_get_pad_format(&hsit->entity, config, fse->pad);
-
-	if (fse->index || fse->code != format->code)
-		return -EINVAL;
-
-	if (fse->pad == HSIT_PAD_SINK) {
-		fse->min_width = HSIT_MIN_SIZE;
-		fse->max_width = HSIT_MAX_SIZE;
-		fse->min_height = HSIT_MIN_SIZE;
-		fse->max_height = HSIT_MAX_SIZE;
-	} else {
-		/* The size on the source pad are fixed and always identical to
-		 * the size on the sink pad.
-		 */
-		fse->min_width = format->width;
-		fse->max_width = format->width;
-		fse->min_height = format->height;
-		fse->max_height = format->height;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_frame_size(subdev, cfg, fse, HSIT_MIN_SIZE,
+					   HSIT_MIN_SIZE, HSIT_MAX_SIZE,
+					   HSIT_MAX_SIZE);
 }
 
 static int hsit_set_format(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index ded4c5a87d4c..d1d52a25c15b 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -54,32 +54,9 @@ static int lif_enum_frame_size(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_pad_config *cfg,
 			       struct v4l2_subdev_frame_size_enum *fse)
 {
-	struct vsp1_lif *lif = to_lif(subdev);
-	struct v4l2_subdev_pad_config *config;
-	struct v4l2_mbus_framefmt *format;
-
-	config = vsp1_entity_get_pad_config(&lif->entity, cfg, fse->which);
-	if (!config)
-		return -EINVAL;
-
-	format = vsp1_entity_get_pad_format(&lif->entity, config, LIF_PAD_SINK);
-
-	if (fse->index || fse->code != format->code)
-		return -EINVAL;
-
-	if (fse->pad == LIF_PAD_SINK) {
-		fse->min_width = LIF_MIN_SIZE;
-		fse->max_width = LIF_MAX_SIZE;
-		fse->min_height = LIF_MIN_SIZE;
-		fse->max_height = LIF_MAX_SIZE;
-	} else {
-		fse->min_width = format->width;
-		fse->max_width = format->width;
-		fse->min_height = format->height;
-		fse->max_height = format->height;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_frame_size(subdev, cfg, fse, LIF_MIN_SIZE,
+					   LIF_MIN_SIZE, LIF_MAX_SIZE,
+					   LIF_MAX_SIZE);
 }
 
 static int lif_set_format(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 781ea99abccf..875937681850 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -80,35 +80,9 @@ static int lut_enum_frame_size(struct v4l2_subdev *subdev,
 			       struct v4l2_subdev_pad_config *cfg,
 			       struct v4l2_subdev_frame_size_enum *fse)
 {
-	struct vsp1_lut *lut = to_lut(subdev);
-	struct v4l2_subdev_pad_config *config;
-	struct v4l2_mbus_framefmt *format;
-
-	config = vsp1_entity_get_pad_config(&lut->entity, cfg, fse->which);
-	if (!config)
-		return -EINVAL;
-
-	format = vsp1_entity_get_pad_format(&lut->entity, config, fse->pad);
-
-	if (fse->index || fse->code != format->code)
-		return -EINVAL;
-
-	if (fse->pad == LUT_PAD_SINK) {
-		fse->min_width = LUT_MIN_SIZE;
-		fse->max_width = LUT_MAX_SIZE;
-		fse->min_height = LUT_MIN_SIZE;
-		fse->max_height = LUT_MAX_SIZE;
-	} else {
-		/* The size on the source pad are fixed and always identical to
-		 * the size on the sink pad.
-		 */
-		fse->min_width = format->width;
-		fse->max_width = format->width;
-		fse->min_height = format->height;
-		fse->max_height = format->height;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_frame_size(subdev, cfg, fse, LUT_MIN_SIZE,
+					   LUT_MIN_SIZE, LUT_MAX_SIZE,
+					   LUT_MAX_SIZE);
 }
 
 static int lut_set_format(struct v4l2_subdev *subdev,
diff --git a/drivers/media/platform/vsp1/vsp1_rwpf.c b/drivers/media/platform/vsp1/vsp1_rwpf.c
index 64d649a1bcf5..3b6e032e7806 100644
--- a/drivers/media/platform/vsp1/vsp1_rwpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rwpf.c
@@ -53,34 +53,10 @@ static int vsp1_rwpf_enum_frame_size(struct v4l2_subdev *subdev,
 				     struct v4l2_subdev_frame_size_enum *fse)
 {
 	struct vsp1_rwpf *rwpf = to_rwpf(subdev);
-	struct v4l2_subdev_pad_config *config;
-	struct v4l2_mbus_framefmt *format;
-
-	config = vsp1_entity_get_pad_config(&rwpf->entity, cfg, fse->which);
-	if (!config)
-		return -EINVAL;
-
-	format = vsp1_entity_get_pad_format(&rwpf->entity, config, fse->pad);
 
-	if (fse->index || fse->code != format->code)
-		return -EINVAL;
-
-	if (fse->pad == RWPF_PAD_SINK) {
-		fse->min_width = RWPF_MIN_WIDTH;
-		fse->max_width = rwpf->max_width;
-		fse->min_height = RWPF_MIN_HEIGHT;
-		fse->max_height = rwpf->max_height;
-	} else {
-		/* The size on the source pad are fixed and always identical to
-		 * the size on the sink pad.
-		 */
-		fse->min_width = format->width;
-		fse->max_width = format->width;
-		fse->min_height = format->height;
-		fse->max_height = format->height;
-	}
-
-	return 0;
+	return vsp1_subdev_enum_frame_size(subdev, cfg, fse, RWPF_MIN_WIDTH,
+					   RWPF_MIN_HEIGHT, rwpf->max_width,
+					   rwpf->max_height);
 }
 
 static int vsp1_rwpf_set_format(struct v4l2_subdev *subdev,
-- 
2.7.3

[PATCH v2 45/54] v4l: vsp1: Fix LUT format setting

[Laurent Pinchart]

The LUT set format handler overrides the requested format by mistake.
Fix it.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_lut.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index 875937681850..d45f563dea00 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -111,6 +111,7 @@ static int lut_set_format(struct v4l2_subdev *subdev,
 		return 0;
 	}
 
+	format->code = fmt->format.code;
 	format->width = clamp_t(unsigned int, fmt->format.width,
 				LUT_MIN_SIZE, LUT_MAX_SIZE);
 	format->height = clamp_t(unsigned int, fmt->format.height,
-- 
2.7.3

[PATCH v2 46/54] v4l: vsp1: dl: Make reg_count field unsigned

[Laurent Pinchart]

The field takes positive values only, make it unsigned.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_dl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 5a24d494059b..8efa5447c1b3 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -60,7 +60,7 @@ struct vsp1_dl_list {
 	dma_addr_t dma;
 	size_t size;
 
-	int reg_count;
+	unsigned int reg_count;
 };
 
 enum vsp1_dl_mode {
-- 
2.7.3

[PATCH v2 47/54] v4l: vsp1: dl: Fix race conditions

[Laurent Pinchart]

The vsp1_dl_list_put() function expects to be called with the display
list manager lock held. This assumption is correct for calls from within
the vsp1_dl.c file, but not for the external calls. Fix it by taking the
lock inside the function and providing an unlocked version for the
internal callers.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_dl.c | 41 +++++++++++++++++++++++++----------
 1 file changed, 29 insertions(+), 12 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 8efa5447c1b3..4f2c3c95bfa4 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -164,25 +164,36 @@ struct vsp1_dl_list *vsp1_dl_list_get(struct vsp1_dl_manager *dlm)
 	return dl;
 }
 
+/* This function must be called with the display list manager lock held.*/
+static void __vsp1_dl_list_put(struct vsp1_dl_list *dl)
+{
+	if (!dl)
+		return;
+
+	dl->reg_count = 0;
+
+	list_add_tail(&dl->list, &dl->dlm->free);
+}
+
 /**
  * vsp1_dl_list_put - Release a display list
  * @dl: The display list
  *
  * Release the display list and return it to the pool of free lists.
  *
- * This function must be called with the display list manager lock held.
- *
  * Passing a NULL pointer to this function is safe, in that case no operation
  * will be performed.
  */
 void vsp1_dl_list_put(struct vsp1_dl_list *dl)
 {
+	unsigned long flags;
+
 	if (!dl)
 		return;
 
-	dl->reg_count = 0;
-
-	list_add_tail(&dl->list, &dl->dlm->free);
+	spin_lock_irqsave(&dl->dlm->lock, flags);
+	__vsp1_dl_list_put(dl);
+	spin_unlock_irqrestore(&dl->dlm->lock, flags);
 }
 
 void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg, u32 data)
@@ -220,7 +231,7 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
 	 */
 	update = !!(vsp1_read(vsp1, VI6_DL_BODY_SIZE) & VI6_DL_BODY_SIZE_UPD);
 	if (update) {
-		vsp1_dl_list_put(dlm->pending);
+		__vsp1_dl_list_put(dlm->pending);
 		dlm->pending = dl;
 		goto done;
 	}
@@ -233,7 +244,7 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
 	vsp1_write(vsp1, VI6_DL_BODY_SIZE, VI6_DL_BODY_SIZE_UPD |
 		   (dl->reg_count * 8));
 
-	vsp1_dl_list_put(dlm->queued);
+	__vsp1_dl_list_put(dlm->queued);
 	dlm->queued = dl;
 
 done:
@@ -253,7 +264,7 @@ void vsp1_dlm_irq_display_start(struct vsp1_dl_manager *dlm)
 	 * processing by the device. The active display list, if any, won't be
 	 * accessed anymore and can be reused.
 	 */
-	vsp1_dl_list_put(dlm->active);
+	__vsp1_dl_list_put(dlm->active);
 	dlm->active = NULL;
 
 	spin_unlock(&dlm->lock);
@@ -265,7 +276,7 @@ void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
 
 	spin_lock(&dlm->lock);
 
-	vsp1_dl_list_put(dlm->active);
+	__vsp1_dl_list_put(dlm->active);
 	dlm->active = NULL;
 
 	/* Header mode is used for mem-to-mem pipelines only. We don't need to
@@ -328,9 +339,15 @@ void vsp1_dlm_setup(struct vsp1_device *vsp1)
 
 void vsp1_dlm_reset(struct vsp1_dl_manager *dlm)
 {
-	vsp1_dl_list_put(dlm->active);
-	vsp1_dl_list_put(dlm->queued);
-	vsp1_dl_list_put(dlm->pending);
+	unsigned long flags;
+
+	spin_lock_irqsave(&dlm->lock, flags);
+
+	__vsp1_dl_list_put(dlm->active);
+	__vsp1_dl_list_put(dlm->queued);
+	__vsp1_dl_list_put(dlm->pending);
+
+	spin_unlock_irqrestore(&dlm->lock, flags);
 
 	dlm->active = NULL;
 	dlm->queued = NULL;
-- 
2.7.3

[PATCH v2 48/54] v4l: vsp1: dl: Add support for multi-body display lists

[Laurent Pinchart]

Display lists support up to 8 bodies but we currently use a single one.
To support preparing display lists for large look-up tables, add support
for multi-body display lists.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_dl.c | 291 ++++++++++++++++++++++++++++------
 drivers/media/platform/vsp1/vsp1_dl.h |   8 +
 2 files changed, 252 insertions(+), 47 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_dl.c b/drivers/media/platform/vsp1/vsp1_dl.c
index 4f2c3c95bfa4..f283035560a8 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.c
+++ b/drivers/media/platform/vsp1/vsp1_dl.c
@@ -19,28 +19,20 @@
 #include "vsp1.h"
 #include "vsp1_dl.h"
 
-/*
- * Global resources
- *
- * - Display-related interrupts (can be used for vblank evasion ?)
- * - Display-list enable
- * - Header-less for WPF0
- * - DL swap
- */
-
-#define VSP1_DL_HEADER_SIZE		76
-#define VSP1_DL_BODY_SIZE		(2 * 4 * 256)
+#define VSP1_DL_NUM_ENTRIES		256
 #define VSP1_DL_NUM_LISTS		3
 
 #define VSP1_DLH_INT_ENABLE		(1 << 1)
 #define VSP1_DLH_AUTO_START		(1 << 0)
 
+struct vsp1_dl_header_list {
+	u32 num_bytes;
+	u32 addr;
+} __attribute__((__packed__));
+
 struct vsp1_dl_header {
 	u32 num_lists;
-	struct {
-		u32 num_bytes;
-		u32 addr;
-	} lists[8];
+	struct vsp1_dl_header_list lists[8];
 	u32 next_header;
 	u32 flags;
 } __attribute__((__packed__));
@@ -50,17 +42,44 @@ struct vsp1_dl_entry {
 	u32 data;
 } __attribute__((__packed__));
 
-struct vsp1_dl_list {
+/**
+ * struct vsp1_dl_body - Display list body
+ * @list: entry in the display list list of bodies
+ * @vsp1: the VSP1 device
+ * @entries: array of entries
+ * @dma: DMA address of the entries
+ * @size: size of the DMA memory in bytes
+ * @num_entries: number of stored entries
+ */
+struct vsp1_dl_body {
 	struct list_head list;
+	struct vsp1_device *vsp1;
+
+	struct vsp1_dl_entry *entries;
+	dma_addr_t dma;
+	size_t size;
+
+	unsigned int num_entries;
+};
 
+/**
+ * struct vsp1_dl_list - Display list
+ * @list: entry in the display list manager lists
+ * @dlm: the display list manager
+ * @header: display list header, NULL for headerless lists
+ * @dma: DMA address for the header
+ * @body0: first display list body
+ * @fragments: list of extra display list bodies
+ */
+struct vsp1_dl_list {
+	struct list_head list;
 	struct vsp1_dl_manager *dlm;
 
 	struct vsp1_dl_header *header;
-	struct vsp1_dl_entry *body;
 	dma_addr_t dma;
-	size_t size;
 
-	unsigned int reg_count;
+	struct vsp1_dl_body body0;
+	struct list_head fragments;
 };
 
 enum vsp1_dl_mode {
@@ -92,6 +111,111 @@ struct vsp1_dl_manager {
 };
 
 /* -----------------------------------------------------------------------------
+ * Display List Body Management
+ */
+
+/*
+ * Initialize a display list body object and allocate DMA memory for the body
+ * data. The display list body object is expected to have been initialized to
+ * 0 when allocated.
+ */
+static int vsp1_dl_body_init(struct vsp1_device *vsp1,
+			     struct vsp1_dl_body *dlb, unsigned int num_entries,
+			     size_t extra_size)
+{
+	size_t size = num_entries * sizeof(*dlb->entries) + extra_size;
+
+	dlb->vsp1 = vsp1;
+	dlb->size = size;
+
+	dlb->entries = dma_alloc_writecombine(vsp1->dev, dlb->size,
+					      &dlb->dma, GFP_KERNEL);
+	if (!dlb->entries)
+		return -ENOMEM;
+
+	return 0;
+}
+
+/*
+ * Cleanup a display list body and free allocated DMA memory allocated.
+ */
+static void vsp1_dl_body_cleanup(struct vsp1_dl_body *dlb)
+{
+	dma_free_writecombine(dlb->vsp1->dev, dlb->size, dlb->entries,
+			      dlb->dma);
+}
+
+/**
+ * vsp1_dl_fragment_alloc - Allocate a display list fragment
+ * @vsp1: The VSP1 device
+ * @num_entries: The maximum number of entries that the fragment can contain
+ *
+ * Allocate a display list fragment with enough memory to contain the requested
+ * number of entries.
+ *
+ * Return a pointer to a fragment on success or NULL if memory can't be
+ * allocated.
+ */
+struct vsp1_dl_body *vsp1_dl_fragment_alloc(struct vsp1_device *vsp1,
+					    unsigned int num_entries)
+{
+	struct vsp1_dl_body *dlb;
+	int ret;
+
+	dlb = kzalloc(sizeof(*dlb), GFP_KERNEL);
+	if (!dlb)
+		return NULL;
+
+	ret = vsp1_dl_body_init(vsp1, dlb, num_entries, 0);
+	if (ret < 0) {
+		kfree(dlb);
+		return NULL;
+	}
+
+	return dlb;
+}
+
+/**
+ * vsp1_dl_fragment_free - Free a display list fragment
+ * @dlb: The fragment
+ *
+ * Free the given display list fragment and the associated DMA memory.
+ *
+ * Fragments must only be freed explicitly if they are not added to a display
+ * list, as the display list will take ownership of them and free them
+ * otherwise. Manual free typically happens at cleanup time for fragments that
+ * have been allocated but not used.
+ *
+ * Passing a NULL pointer to this function is safe, in that case no operation
+ * will be performed.
+ */
+void vsp1_dl_fragment_free(struct vsp1_dl_body *dlb)
+{
+	if (!dlb)
+		return;
+
+	vsp1_dl_body_cleanup(dlb);
+	kfree(dlb);
+}
+
+/**
+ * vsp1_dl_fragment_write - Write a register to a display list fragment
+ * @dlb: The fragment
+ * @reg: The register address
+ * @data: The register value
+ *
+ * Write the given register and value to the display list fragment. The maximum
+ * number of entries that can be written in a fragment is specified when the
+ * fragment is allocated by vsp1_dl_fragment_alloc().
+ */
+void vsp1_dl_fragment_write(struct vsp1_dl_body *dlb, u32 reg, u32 data)
+{
+	dlb->entries[dlb->num_entries].addr = reg;
+	dlb->entries[dlb->num_entries].data = data;
+	dlb->num_entries++;
+}
+
+/* -----------------------------------------------------------------------------
  * Display List Transaction Management
  */
 
@@ -99,43 +223,61 @@ static struct vsp1_dl_list *vsp1_dl_list_alloc(struct vsp1_dl_manager *dlm)
 {
 	struct vsp1_dl_list *dl;
 	size_t header_size;
-
-	/* The body needs to be aligned on a 8 bytes boundary, pad the header
-	 * size to allow allocating both in a single operation.
-	 */
-	header_size = dlm->mode == VSP1_DL_MODE_HEADER
-		    ? ALIGN(sizeof(struct vsp1_dl_header), 8)
-		    : 0;
+	int ret;
 
 	dl = kzalloc(sizeof(*dl), GFP_KERNEL);
 	if (!dl)
 		return NULL;
 
+	INIT_LIST_HEAD(&dl->fragments);
 	dl->dlm = dlm;
-	dl->size = header_size + VSP1_DL_BODY_SIZE;
 
-	dl->header = dma_alloc_writecombine(dlm->vsp1->dev, dl->size, &dl->dma,
-					    GFP_KERNEL);
-	if (!dl->header) {
+	/* Initialize the display list body and allocate DMA memory for the body
+	 * and the optional header. Both are allocated together to avoid memory
+	 * fragmentation, with the header located right after the body in
+	 * memory.
+	 */
+	header_size = dlm->mode == VSP1_DL_MODE_HEADER
+		    ? ALIGN(sizeof(struct vsp1_dl_header), 8)
+		    : 0;
+
+	ret = vsp1_dl_body_init(dlm->vsp1, &dl->body0, VSP1_DL_NUM_ENTRIES,
+				header_size);
+	if (ret < 0) {
 		kfree(dl);
 		return NULL;
 	}
 
 	if (dlm->mode == VSP1_DL_MODE_HEADER) {
+		size_t header_offset = VSP1_DL_NUM_ENTRIES
+				     * sizeof(*dl->body0.entries);
+
+		dl->header = ((void *)dl->body0.entries) + header_offset;
+		dl->dma = dl->body0.dma + header_offset;
+
 		memset(dl->header, 0, sizeof(*dl->header));
-		dl->header->lists[0].addr = dl->dma + header_size;
+		dl->header->lists[0].addr = dl->body0.dma;
 		dl->header->flags = VSP1_DLH_INT_ENABLE;
 	}
 
-	dl->body = ((void *)dl->header) + header_size;
-
 	return dl;
 }
 
+static void vsp1_dl_list_free_fragments(struct vsp1_dl_list *dl)
+{
+	struct vsp1_dl_body *dlb, *next;
+
+	list_for_each_entry_safe(dlb, next, &dl->fragments, list) {
+		list_del(&dlb->list);
+		vsp1_dl_body_cleanup(dlb);
+		kfree(dlb);
+	}
+}
+
 static void vsp1_dl_list_free(struct vsp1_dl_list *dl)
 {
-	dma_free_writecombine(dl->dlm->vsp1->dev, dl->size, dl->header,
-			      dl->dma);
+	vsp1_dl_body_cleanup(&dl->body0);
+	vsp1_dl_list_free_fragments(dl);
 	kfree(dl);
 }
 
@@ -170,7 +312,8 @@ static void __vsp1_dl_list_put(struct vsp1_dl_list *dl)
 	if (!dl)
 		return;
 
-	dl->reg_count = 0;
+	vsp1_dl_list_free_fragments(dl);
+	dl->body0.num_entries = 0;
 
 	list_add_tail(&dl->list, &dl->dlm->free);
 }
@@ -196,11 +339,45 @@ void vsp1_dl_list_put(struct vsp1_dl_list *dl)
 	spin_unlock_irqrestore(&dl->dlm->lock, flags);
 }
 
+/**
+ * vsp1_dl_list_write - Write a register to the display list
+ * @dl: The display list
+ * @reg: The register address
+ * @data: The register value
+ *
+ * Write the given register and value to the display list. Up to 256 registers
+ * can be written per display list.
+ */
 void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg, u32 data)
 {
-	dl->body[dl->reg_count].addr = reg;
-	dl->body[dl->reg_count].data = data;
-	dl->reg_count++;
+	vsp1_dl_fragment_write(&dl->body0, reg, data);
+}
+
+/**
+ * vsp1_dl_list_add_fragment - Add a fragment to the display list
+ * @dl: The display list
+ * @dlb: The fragment
+ *
+ * Add a display list body as a fragment to a display list. Registers contained
+ * in fragments are processed after registers contained in the main display
+ * list, in the order in which fragments are added.
+ *
+ * Adding a fragment to a display list passes ownership of the fragment to the
+ * list. The caller must not touch the fragment after this call, and must not
+ * free it explicitly with vsp1_dl_fragment_free().
+ *
+ * Fragments are only usable for display lists in header mode. Attempt to
+ * add a fragment to a header-less display list will return an error.
+ */
+int vsp1_dl_list_add_fragment(struct vsp1_dl_list *dl,
+			      struct vsp1_dl_body *dlb)
+{
+	/* Multi-body lists are only available in header mode. */
+	if (dl->dlm->mode != VSP1_DL_MODE_HEADER)
+		return -EINVAL;
+
+	list_add_tail(&dlb->list, &dl->fragments);
+	return 0;
 }
 
 void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
@@ -213,11 +390,30 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
 	spin_lock_irqsave(&dlm->lock, flags);
 
 	if (dl->dlm->mode == VSP1_DL_MODE_HEADER) {
-		/* Program the hardware with the display list body address and
-		 * size. In header mode the caller guarantees that the hardware
-		 * is idle at this point.
+		struct vsp1_dl_header_list *hdr = dl->header->lists;
+		struct vsp1_dl_body *dlb;
+		unsigned int num_lists = 0;
+
+		/* Fill the header with the display list bodies addresses and
+		 * sizes. The address of the first body has already been filled
+		 * when the display list was allocated.
+		 *
+		 * In header mode the caller guarantees that the hardware is
+		 * idle at this point.
 		 */
-		dl->header->lists[0].num_bytes = dl->reg_count * 8;
+		hdr->num_bytes = dl->body0.num_entries
+			       * sizeof(*dl->header->lists);
+
+		list_for_each_entry(dlb, &dl->fragments, list) {
+			num_lists++;
+			hdr++;
+
+			hdr->addr = dlb->dma;
+			hdr->num_bytes = dlb->num_entries
+				       * sizeof(*dl->header->lists);
+		}
+
+		dl->header->num_lists = num_lists;
 		vsp1_write(vsp1, VI6_DL_HDR_ADDR(dlm->index), dl->dma);
 
 		dlm->active = dl;
@@ -240,9 +436,9 @@ void vsp1_dl_list_commit(struct vsp1_dl_list *dl)
 	 * The UPD bit will be cleared by the device when the display list is
 	 * processed.
 	 */
-	vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->dma);
+	vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->body0.dma);
 	vsp1_write(vsp1, VI6_DL_BODY_SIZE, VI6_DL_BODY_SIZE_UPD |
-		   (dl->reg_count * 8));
+		   (dl->body0.num_entries * sizeof(*dl->header->lists)));
 
 	__vsp1_dl_list_put(dlm->queued);
 	dlm->queued = dl;
@@ -308,9 +504,10 @@ void vsp1_dlm_irq_frame_end(struct vsp1_dl_manager *dlm)
 	if (dlm->pending) {
 		struct vsp1_dl_list *dl = dlm->pending;
 
-		vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->dma);
+		vsp1_write(vsp1, VI6_DL_HDR_ADDR(0), dl->body0.dma);
 		vsp1_write(vsp1, VI6_DL_BODY_SIZE, VI6_DL_BODY_SIZE_UPD |
-			   (dl->reg_count * 8));
+			   (dl->body0.num_entries *
+			    sizeof(*dl->header->lists)));
 
 		dlm->queued = dl;
 		dlm->pending = NULL;
diff --git a/drivers/media/platform/vsp1/vsp1_dl.h b/drivers/media/platform/vsp1/vsp1_dl.h
index 571ed6d8e7c2..de387cd4d745 100644
--- a/drivers/media/platform/vsp1/vsp1_dl.h
+++ b/drivers/media/platform/vsp1/vsp1_dl.h
@@ -16,6 +16,7 @@
 #include <linux/types.h>
 
 struct vsp1_device;
+struct vsp1_dl_fragment;
 struct vsp1_dl_list;
 struct vsp1_dl_manager;
 
@@ -34,4 +35,11 @@ void vsp1_dl_list_put(struct vsp1_dl_list *dl);
 void vsp1_dl_list_write(struct vsp1_dl_list *dl, u32 reg, u32 data);
 void vsp1_dl_list_commit(struct vsp1_dl_list *dl);
 
+struct vsp1_dl_body *vsp1_dl_fragment_alloc(struct vsp1_device *vsp1,
+					    unsigned int num_entries);
+void vsp1_dl_fragment_free(struct vsp1_dl_body *dlb);
+void vsp1_dl_fragment_write(struct vsp1_dl_body *dlb, u32 reg, u32 data);
+int vsp1_dl_list_add_fragment(struct vsp1_dl_list *dl,
+			      struct vsp1_dl_body *dlb);
+
 #endif /* __VSP1_DL_H__ */
-- 
2.7.3

[PATCH v2 49/54] v4l: vsp1: lut: Use display list fragments to fill LUT

[Laurent Pinchart]

Synchronize the userspace LUT setup with the pipeline operation by using
a display list fragment to store LUT data.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_lut.c | 31 ++++++++++++++++++++++++++-----
 drivers/media/platform/vsp1/vsp1_lut.h |  6 +++++-
 2 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_lut.c b/drivers/media/platform/vsp1/vsp1_lut.c
index d45f563dea00..c5250abfd0d0 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.c
+++ b/drivers/media/platform/vsp1/vsp1_lut.c
@@ -38,10 +38,25 @@ static inline void vsp1_lut_write(struct vsp1_lut *lut, struct vsp1_dl_list *dl,
  * V4L2 Subdevice Core Operations
  */
 
-static void lut_set_table(struct vsp1_lut *lut, struct vsp1_lut_config *config)
+static int lut_set_table(struct vsp1_lut *lut, struct vsp1_lut_config *config)
 {
-	memcpy_toio(lut->entity.vsp1->mmio + VI6_LUT_TABLE, config->lut,
-		    sizeof(config->lut));
+	struct vsp1_dl_body *dlb;
+	unsigned int i;
+
+	dlb = vsp1_dl_fragment_alloc(lut->entity.vsp1, ARRAY_SIZE(config->lut));
+	if (!dlb)
+		return -ENOMEM;
+
+	for (i = 0; i < ARRAY_SIZE(config->lut); ++i)
+		vsp1_dl_fragment_write(dlb, VI6_LUT_TABLE + 4 * i,
+				       config->lut[i]);
+
+	mutex_lock(&lut->lock);
+	swap(lut->lut, dlb);
+	mutex_unlock(&lut->lock);
+
+	vsp1_dl_fragment_free(dlb);
+	return 0;
 }
 
 static long lut_ioctl(struct v4l2_subdev *subdev, unsigned int cmd, void *arg)
@@ -50,8 +65,7 @@ static long lut_ioctl(struct v4l2_subdev *subdev, unsigned int cmd, void *arg)
 
 	switch (cmd) {
 	case VIDIOC_VSP1_LUT_CONFIG:
-		lut_set_table(lut, arg);
-		return 0;
+		return lut_set_table(lut, arg);
 
 	default:
 		return -ENOIOCTLCMD;
@@ -161,6 +175,13 @@ static void lut_configure(struct vsp1_entity *entity,
 	struct vsp1_lut *lut = to_lut(&entity->subdev);
 
 	vsp1_lut_write(lut, dl, VI6_LUT_CTRL, VI6_LUT_CTRL_EN);
+
+	mutex_lock(&lut->lock);
+	if (lut->lut) {
+		vsp1_dl_list_add_fragment(dl, lut->lut);
+		lut->lut = NULL;
+	}
+	mutex_unlock(&lut->lock);
 }
 
 static const struct vsp1_entity_operations lut_entity_ops = {
diff --git a/drivers/media/platform/vsp1/vsp1_lut.h b/drivers/media/platform/vsp1/vsp1_lut.h
index f92ffb867350..cef874f22b6a 100644
--- a/drivers/media/platform/vsp1/vsp1_lut.h
+++ b/drivers/media/platform/vsp1/vsp1_lut.h
@@ -13,6 +13,8 @@
 #ifndef __VSP1_LUT_H__
 #define __VSP1_LUT_H__
 
+#include <linux/mutex.h>
+
 #include <media/media-entity.h>
 #include <media/v4l2-subdev.h>
 
@@ -25,7 +27,9 @@ struct vsp1_device;
 
 struct vsp1_lut {
 	struct vsp1_entity entity;
-	u32 lut[256];
+
+	struct mutex lock;
+	struct vsp1_dl_body *lut;
 };
 
 static inline struct vsp1_lut *to_lut(struct v4l2_subdev *subdev)
-- 
2.7.3

[PATCH v2 50/54] v4l: vsp1: Add support for the RPF alpha multiplier on Gen3

[Laurent Pinchart]

The Gen3 RPF includes an alpha multiplier that can both multiply the
alpha channel by a fixed global alpha value, and multiply the pixel
components to convert the input to premultiplied alpha.

As alpha premultiplication is available in the BRU for both Gen2 and
Gen3 we handle it there and use the Gen3 alpha multiplier for global
alpha multiplication only. This prevents conversion to premultiplied
alpha if no BRU is present in the pipeline, that use case will be
implemented later if needed.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1.h      |  1 +
 drivers/media/platform/vsp1/vsp1_drv.c  |  8 +++++
 drivers/media/platform/vsp1/vsp1_regs.h | 10 ++++++
 drivers/media/platform/vsp1/vsp1_rpf.c  | 57 +++++++++++++++++++++++++++++++--
 4 files changed, 73 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1.h b/drivers/media/platform/vsp1/vsp1.h
index dae987a11a70..46738b6c5f72 100644
--- a/drivers/media/platform/vsp1/vsp1.h
+++ b/drivers/media/platform/vsp1/vsp1.h
@@ -48,6 +48,7 @@ struct vsp1_uds;
 
 struct vsp1_device_info {
 	u32 version;
+	unsigned int gen;
 	unsigned int features;
 	unsigned int rpf_count;
 	unsigned int uds_count;
diff --git a/drivers/media/platform/vsp1/vsp1_drv.c b/drivers/media/platform/vsp1/vsp1_drv.c
index 596f26d81494..e2d779fac0eb 100644
--- a/drivers/media/platform/vsp1/vsp1_drv.c
+++ b/drivers/media/platform/vsp1/vsp1_drv.c
@@ -558,6 +558,7 @@ static const struct dev_pm_ops vsp1_pm_ops = {
 static const struct vsp1_device_info vsp1_device_infos[] = {
 	{
 		.version = VI6_IP_VERSION_MODEL_VSPS_H2,
+		.gen = 2,
 		.features = VSP1_HAS_BRU | VSP1_HAS_LUT | VSP1_HAS_SRU,
 		.rpf_count = 5,
 		.uds_count = 3,
@@ -566,6 +567,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPR_H2,
+		.gen = 2,
 		.features = VSP1_HAS_BRU | VSP1_HAS_SRU,
 		.rpf_count = 5,
 		.uds_count = 1,
@@ -574,6 +576,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPD_GEN2,
+		.gen = 2,
 		.features = VSP1_HAS_BRU | VSP1_HAS_LIF | VSP1_HAS_LUT,
 		.rpf_count = 4,
 		.uds_count = 1,
@@ -582,6 +585,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPS_M2,
+		.gen = 2,
 		.features = VSP1_HAS_BRU | VSP1_HAS_LUT | VSP1_HAS_SRU,
 		.rpf_count = 5,
 		.uds_count = 3,
@@ -590,6 +594,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPI_GEN3,
+		.gen = 3,
 		.features = VSP1_HAS_LUT | VSP1_HAS_SRU,
 		.rpf_count = 1,
 		.uds_count = 1,
@@ -597,6 +602,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPBD_GEN3,
+		.gen = 3,
 		.features = VSP1_HAS_BRU,
 		.rpf_count = 5,
 		.wpf_count = 1,
@@ -604,6 +610,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPBC_GEN3,
+		.gen = 3,
 		.features = VSP1_HAS_BRU | VSP1_HAS_LUT,
 		.rpf_count = 5,
 		.wpf_count = 1,
@@ -611,6 +618,7 @@ static const struct vsp1_device_info vsp1_device_infos[] = {
 		.uapi = true,
 	}, {
 		.version = VI6_IP_VERSION_MODEL_VSPD_GEN3,
+		.gen = 3,
 		.features = VSP1_HAS_BRU | VSP1_HAS_LIF,
 		.rpf_count = 5,
 		.wpf_count = 2,
diff --git a/drivers/media/platform/vsp1/vsp1_regs.h b/drivers/media/platform/vsp1/vsp1_regs.h
index 069216f0eb44..927b5fb94c48 100644
--- a/drivers/media/platform/vsp1/vsp1_regs.h
+++ b/drivers/media/platform/vsp1/vsp1_regs.h
@@ -217,6 +217,16 @@
 #define VI6_RPF_SRCM_ADDR_C1		0x0344
 #define VI6_RPF_SRCM_ADDR_AI		0x0348
 
+#define VI6_RPF_MULT_ALPHA		0x036c
+#define VI6_RPF_MULT_ALPHA_A_MMD_NONE	(0 << 12)
+#define VI6_RPF_MULT_ALPHA_A_MMD_RATIO	(1 << 12)
+#define VI6_RPF_MULT_ALPHA_P_MMD_NONE	(0 << 8)
+#define VI6_RPF_MULT_ALPHA_P_MMD_RATIO	(1 << 8)
+#define VI6_RPF_MULT_ALPHA_P_MMD_IMAGE	(2 << 8)
+#define VI6_RPF_MULT_ALPHA_P_MMD_BOTH	(3 << 8)
+#define VI6_RPF_MULT_ALPHA_RATIO_MASK	(0xff < 0)
+#define VI6_RPF_MULT_ALPHA_RATIO_SHIFT	0
+
 /* -----------------------------------------------------------------------------
  * WPF Control Registers
  */
diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index bc94427c0740..a316e1030610 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -141,9 +141,27 @@ static void rpf_configure(struct vsp1_entity *entity,
 		       (left << VI6_RPF_LOC_HCOORD_SHIFT) |
 		       (top << VI6_RPF_LOC_VCOORD_SHIFT));
 
-	/* Use the alpha channel (extended to 8 bits) when available or an
-	 * alpha value set through the V4L2_CID_ALPHA_COMPONENT control
-	 * otherwise. Disable color keying.
+	/* On Gen2 use the alpha channel (extended to 8 bits) when available or
+	 * a fixed alpha value set through the V4L2_CID_ALPHA_COMPONENT control
+	 * otherwise.
+	 *
+	 * The Gen3 RPF has extended alpha capability and can both multiply the
+	 * alpha channel by a fixed global alpha value, and multiply the pixel
+	 * components to convert the input to premultiplied alpha.
+	 *
+	 * As alpha premultiplication is available in the BRU for both Gen2 and
+	 * Gen3 we handle it there and use the Gen3 alpha multiplier for global
+	 * alpha multiplication only. This however prevents conversion to
+	 * premultiplied alpha if no BRU is present in the pipeline. If that use
+	 * case turns out to be useful we will revisit the implementation (for
+	 * Gen3 only).
+	 *
+	 * We enable alpha multiplication on Gen3 using the fixed alpha value
+	 * set through the V4L2_CID_ALPHA_COMPONENT control when the input
+	 * contains an alpha channel. On Gen2 the global alpha is ignored in
+	 * that case.
+	 *
+	 * In all cases, disable color keying.
 	 */
 	vsp1_rpf_write(rpf, dl, VI6_RPF_ALPH_SEL, VI6_RPF_ALPH_SEL_AEXT_EXT |
 		       (fmtinfo->alpha ? VI6_RPF_ALPH_SEL_ASEL_PACKED
@@ -152,10 +170,43 @@ static void rpf_configure(struct vsp1_entity *entity,
 	vsp1_rpf_write(rpf, dl, VI6_RPF_VRTCOL_SET,
 		       rpf->alpha << VI6_RPF_VRTCOL_SET_LAYA_SHIFT);
 
+	if (entity->vsp1->info->gen == 3) {
+		u32 mult;
+
+		if (fmtinfo->alpha) {
+			/* When the input contains an alpha channel enable the
+			 * alpha multiplier. If the input is premultiplied we
+			 * need to multiply both the alpha channel and the pixel
+			 * components by the global alpha value to keep them
+			 * premultiplied. Otherwise multiply the alpha channel
+			 * only.
+			 */
+			bool premultiplied = format->flags
+					   & V4L2_PIX_FMT_FLAG_PREMUL_ALPHA;
+
+			mult = VI6_RPF_MULT_ALPHA_A_MMD_RATIO
+			     | (premultiplied ?
+				VI6_RPF_MULT_ALPHA_P_MMD_RATIO :
+				VI6_RPF_MULT_ALPHA_P_MMD_NONE)
+			     | (rpf->alpha << VI6_RPF_MULT_ALPHA_RATIO_SHIFT);
+		} else {
+			/* When the input doesn't contain an alpha channel the
+			 * global alpha value is applied in the unpacking unit,
+			 * the alpha multiplier isn't needed and must be
+			 * disabled.
+			 */
+			mult = VI6_RPF_MULT_ALPHA_A_MMD_NONE
+			     | VI6_RPF_MULT_ALPHA_P_MMD_NONE;
+		}
+
+		vsp1_rpf_write(rpf, dl, VI6_RPF_MULT_ALPHA, mult);
+	}
+
 	vsp1_pipeline_propagate_alpha(pipe, &rpf->entity, dl, rpf->alpha);
 
 	vsp1_rpf_write(rpf, dl, VI6_RPF_MSK_CTRL, 0);
 	vsp1_rpf_write(rpf, dl, VI6_RPF_CKEY_CTRL, 0);
+
 }
 
 static const struct vsp1_entity_operations rpf_entity_ops = {
-- 
2.7.3

[PATCH v2 51/54] v4l: vsp1: Add Z-order support for DRM pipeline

[Laurent Pinchart]

Make the Z-order of planes configurable by assigning RPFs to BRU inputs
dynamically based on the Z-order position.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_drm.c  | 180 +++++++++++++++++++-------------
 drivers/media/platform/vsp1/vsp1_drm.h  |  12 ++-
 drivers/media/platform/vsp1/vsp1_pipe.c |   1 +
 include/media/vsp1.h                    |  18 +++-
 4 files changed, 130 insertions(+), 81 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index 7cde2d970dba..d85cb0e258c9 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -93,6 +93,7 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width,
 		media_entity_pipeline_stop(&pipe->output->entity.subdev.entity);
 
 		for (i = 0; i < bru->entity.source_pad; ++i) {
+			vsp1->drm->inputs[i].enabled = false;
 			bru->inputs[i].rpf = NULL;
 			pipe->inputs[i] = NULL;
 		}
@@ -217,14 +218,9 @@ void vsp1_du_atomic_begin(struct device *dev)
 {
 	struct vsp1_device *vsp1 = dev_get_drvdata(dev);
 	struct vsp1_pipeline *pipe = &vsp1->drm->pipe;
-	unsigned long flags;
-
-	spin_lock_irqsave(&pipe->irqlock, flags);
 
 	vsp1->drm->num_inputs = pipe->num_inputs;
 
-	spin_unlock_irqrestore(&pipe->irqlock, flags);
-
 	/* Prepare the display list. */
 	pipe->dl = vsp1_dl_list_get(pipe->output->dlm);
 }
@@ -239,10 +235,12 @@ EXPORT_SYMBOL_GPL(vsp1_du_atomic_begin);
  * @mem: DMA addresses of the memory buffers (one per plane)
  * @src: the source crop rectangle for the RPF
  * @dst: the destination compose rectangle for the BRU input
+ * @zpos: the Z-order position of the input
  *
  * Configure the VSP to perform composition of the image referenced by @mem
  * through RPF @rpf_index, using the @src crop rectangle and the @dst
- * composition rectangle. The Z-order is fixed with RPF 0 at the bottom.
+ * composition rectangle. The Z-order is configurable with higher @zpos values
+ * displayed on top.
  *
  * Image format as stored in memory is expressed as a V4L2 @pixelformat value.
  * As a special case, setting the pixel format to 0 will disable the RPF. The
@@ -260,24 +258,16 @@ EXPORT_SYMBOL_GPL(vsp1_du_atomic_begin);
  *
  * This function isn't reentrant, the caller needs to serialize calls.
  *
- * TODO: Implement Z-order control by decoupling the RPF index from the BRU
- * input index.
- *
  * Return 0 on success or a negative error code on failure.
  */
-int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
-			  u32 pixelformat, unsigned int pitch,
-			  dma_addr_t mem[2], const struct v4l2_rect *src,
-			  const struct v4l2_rect *dst)
+int vsp1_du_atomic_update_ext(struct device *dev, unsigned int rpf_index,
+			      u32 pixelformat, unsigned int pitch,
+			      dma_addr_t mem[2], const struct v4l2_rect *src,
+			      const struct v4l2_rect *dst, unsigned int zpos)
 {
 	struct vsp1_device *vsp1 = dev_get_drvdata(dev);
-	struct vsp1_pipeline *pipe = &vsp1->drm->pipe;
 	const struct vsp1_format_info *fmtinfo;
-	struct v4l2_subdev_selection sel;
-	struct v4l2_subdev_format format;
 	struct vsp1_rwpf *rpf;
-	unsigned long flags;
-	int ret;
 
 	if (rpf_index >= vsp1->info->rpf_count)
 		return -EINVAL;
@@ -288,31 +278,20 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 		dev_dbg(vsp1->dev, "%s: RPF%u: disable requested\n", __func__,
 			rpf_index);
 
-		spin_lock_irqsave(&pipe->irqlock, flags);
-
-		if (pipe->inputs[rpf_index]) {
-			/* Remove the RPF from the pipeline if it was previously
-			 * enabled.
-			 */
-			vsp1->bru->inputs[rpf_index].rpf = NULL;
-			pipe->inputs[rpf_index] = NULL;
-
-			pipe->num_inputs--;
-		}
-
-		spin_unlock_irqrestore(&pipe->irqlock, flags);
-
+		vsp1->drm->inputs[rpf_index].enabled = false;
 		return 0;
 	}
 
 	dev_dbg(vsp1->dev,
-		"%s: RPF%u: (%u,%u)/%ux%u -> (%u,%u)/%ux%u (%08x), pitch %u dma { %pad, %pad }\n",
+		"%s: RPF%u: (%u,%u)/%ux%u -> (%u,%u)/%ux%u (%08x), pitch %u dma { %pad, %pad } zpos %u\n",
 		__func__, rpf_index,
 		src->left, src->top, src->width, src->height,
 		dst->left, dst->top, dst->width, dst->height,
-		pixelformat, pitch, &mem[0], &mem[1]);
+		pixelformat, pitch, &mem[0], &mem[1], zpos);
 
-	/* Set the stride at the RPF input. */
+	/* Store the format, stride, memory buffer address, crop and compose
+	 * rectangles and Z-order position and for the input.
+	 */
 	fmtinfo = vsp1_get_format_info(pixelformat);
 	if (!fmtinfo) {
 		dev_dbg(vsp1->dev, "Unsupport pixel format %08x for RPF\n",
@@ -325,15 +304,38 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 	rpf->format.plane_fmt[0].bytesperline = pitch;
 	rpf->format.plane_fmt[1].bytesperline = pitch;
 
+	rpf->mem.addr[0] = mem[0];
+	rpf->mem.addr[1] = mem[1];
+	rpf->mem.addr[2] = 0;
+
+	vsp1->drm->inputs[rpf_index].crop = *src;
+	vsp1->drm->inputs[rpf_index].compose = *dst;
+	vsp1->drm->inputs[rpf_index].zpos = zpos;
+	vsp1->drm->inputs[rpf_index].enabled = true;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(vsp1_du_atomic_update_ext);
+
+static int vsp1_du_setup_rpf_pipe(struct vsp1_device *vsp1,
+				  struct vsp1_rwpf *rpf, unsigned int bru_input)
+{
+	struct v4l2_subdev_selection sel;
+	struct v4l2_subdev_format format;
+	const struct v4l2_rect *crop;
+	int ret;
+
 	/* Configure the format on the RPF sink pad and propagate it up to the
 	 * BRU sink pad.
 	 */
+	crop = &vsp1->drm->inputs[rpf->entity.index].crop;
+
 	memset(&format, 0, sizeof(format));
 	format.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	format.pad = RWPF_PAD_SINK;
-	format.format.width = src->width + src->left;
-	format.format.height = src->height + src->top;
-	format.format.code = fmtinfo->mbus;
+	format.format.width = crop->width + crop->left;
+	format.format.height = crop->height + crop->top;
+	format.format.code = rpf->fmtinfo->mbus;
 	format.format.field = V4L2_FIELD_NONE;
 
 	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_fmt, NULL,
@@ -350,7 +352,7 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 	sel.which = V4L2_SUBDEV_FORMAT_ACTIVE;
 	sel.pad = RWPF_PAD_SINK;
 	sel.target = V4L2_SEL_TGT_CROP;
-	sel.r = *src;
+	sel.r = *crop;
 
 	ret = v4l2_subdev_call(&rpf->entity.subdev, pad, set_selection, NULL,
 			       &sel);
@@ -385,7 +387,7 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 		return ret;
 
 	/* BRU sink, propagate the format from the RPF source. */
-	format.pad = rpf->entity.index;
+	format.pad = bru_input;
 
 	ret = v4l2_subdev_call(&vsp1->bru->entity.subdev, pad, set_fmt, NULL,
 			       &format);
@@ -396,9 +398,9 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 		__func__, format.format.width, format.format.height,
 		format.format.code, format.pad);
 
-	sel.pad = rpf->entity.index;
+	sel.pad = bru_input;
 	sel.target = V4L2_SEL_TGT_COMPOSE;
-	sel.r = *dst;
+	sel.r = vsp1->drm->inputs[rpf->entity.index].compose;
 
 	ret = v4l2_subdev_call(&vsp1->bru->entity.subdev, pad, set_selection,
 			       NULL, &sel);
@@ -410,32 +412,13 @@ int vsp1_du_atomic_update(struct device *dev, unsigned int rpf_index,
 		__func__, sel.r.left, sel.r.top, sel.r.width, sel.r.height,
 		sel.pad);
 
-	/* Store the BRU input pad number in the RPF. */
-	rpf->bru_input = rpf->entity.index;
-
-	/* Cache the memory buffer address but don't apply the values to the
-	 * hardware as the crop offsets haven't been computed yet.
-	 */
-	rpf->mem.addr[0] = mem[0];
-	rpf->mem.addr[1] = mem[1];
-	rpf->mem.addr[2] = 0;
-
-	spin_lock_irqsave(&pipe->irqlock, flags);
-
-	/* If the RPF was previously stopped set the BRU input to the RPF and
-	 * store the RPF in the pipeline inputs array.
-	 */
-	if (!pipe->inputs[rpf->entity.index]) {
-		vsp1->bru->inputs[rpf_index].rpf = rpf;
-		pipe->inputs[rpf->entity.index] = rpf;
-		pipe->num_inputs++;
-	}
-
-	spin_unlock_irqrestore(&pipe->irqlock, flags);
-
 	return 0;
 }
-EXPORT_SYMBOL_GPL(vsp1_du_atomic_update);
+
+static unsigned int rpf_zpos(struct vsp1_device *vsp1, struct vsp1_rwpf *rpf)
+{
+	return vsp1->drm->inputs[rpf->entity.index].zpos;
+}
 
 /**
  * vsp1_du_atomic_flush - Commit an atomic update
@@ -445,10 +428,60 @@ void vsp1_du_atomic_flush(struct device *dev)
 {
 	struct vsp1_device *vsp1 = dev_get_drvdata(dev);
 	struct vsp1_pipeline *pipe = &vsp1->drm->pipe;
+	struct vsp1_rwpf *inputs[VSP1_MAX_RPF] = { NULL, };
 	struct vsp1_entity *entity;
 	unsigned long flags;
-	bool stop = false;
+	unsigned int i;
+	int ret;
+
+	/* Count the number of enabled inputs and sort them by Z-order. */
+	pipe->num_inputs = 0;
+
+	for (i = 0; i < vsp1->info->rpf_count; ++i) {
+		struct vsp1_rwpf *rpf = vsp1->rpf[i];
+		unsigned int j;
+
+		if (!vsp1->drm->inputs[i].enabled) {
+			pipe->inputs[i] = NULL;
+			continue;
+		}
+
+		pipe->inputs[i] = rpf;
+
+		/* Insert the RPF in the sorted RPFs array. */
+		for (j = pipe->num_inputs++; j > 0; --j) {
+			if (rpf_zpos(vsp1, inputs[j-1]) <= rpf_zpos(vsp1, rpf))
+				break;
+			inputs[j] = inputs[j-1];
+		}
 
+		inputs[j] = rpf;
+	}
+
+	/* Setup the RPF input pipeline for every enabled input. */
+	for (i = 0; i < vsp1->info->num_bru_inputs; ++i) {
+		struct vsp1_rwpf *rpf = inputs[i];
+
+		if (!rpf) {
+			vsp1->bru->inputs[i].rpf = NULL;
+			continue;
+		}
+
+		vsp1->bru->inputs[i].rpf = rpf;
+		rpf->bru_input = i;
+		rpf->entity.sink_pad = i;
+
+		dev_dbg(vsp1->dev, "%s: connecting RPF.%u to BRU:%u\n",
+			__func__, rpf->entity.index, i);
+
+		ret = vsp1_du_setup_rpf_pipe(vsp1, rpf, i);
+		if (ret < 0)
+			dev_err(vsp1->dev,
+				"%s: failed to setup RPF.%u\n",
+				__func__, rpf->entity.index);
+	}
+
+	/* Configure all entities in the pipeline. */
 	list_for_each_entry(entity, &pipe->entities, list_pipe) {
 		/* Disconnect unused RPFs from the pipeline. */
 		if (entity->type == VSP1_ENTITY_RPF) {
@@ -466,6 +499,9 @@ void vsp1_du_atomic_flush(struct device *dev)
 		if (entity->ops->configure)
 			entity->ops->configure(entity, pipe, pipe->dl);
 
+		/* The memory buffer address must be applied after configuring
+		 * the RPF to make sure the crop offset are computed.
+		 */
 		if (entity->type == VSP1_ENTITY_RPF)
 			vsp1_rwpf_set_memory(to_rwpf(&entity->subdev),
 					     pipe->dl);
@@ -475,19 +511,13 @@ void vsp1_du_atomic_flush(struct device *dev)
 	pipe->dl = NULL;
 
 	/* Start or stop the pipeline if needed. */
-	spin_lock_irqsave(&pipe->irqlock, flags);
-
 	if (!vsp1->drm->num_inputs && pipe->num_inputs) {
 		vsp1_write(vsp1, VI6_DISP_IRQ_STA, 0);
 		vsp1_write(vsp1, VI6_DISP_IRQ_ENB, VI6_DISP_IRQ_ENB_DSTE);
+		spin_lock_irqsave(&pipe->irqlock, flags);
 		vsp1_pipeline_run(pipe);
+		spin_unlock_irqrestore(&pipe->irqlock, flags);
 	} else if (vsp1->drm->num_inputs && !pipe->num_inputs) {
-		stop = true;
-	}
-
-	spin_unlock_irqrestore(&pipe->irqlock, flags);
-
-	if (stop) {
 		vsp1_write(vsp1, VI6_DISP_IRQ_ENB, 0);
 		vsp1_pipeline_stop(pipe);
 	}
diff --git a/drivers/media/platform/vsp1/vsp1_drm.h b/drivers/media/platform/vsp1/vsp1_drm.h
index e9242f2c870e..9e28ab9254ba 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.h
+++ b/drivers/media/platform/vsp1/vsp1_drm.h
@@ -13,18 +13,26 @@
 #ifndef __VSP1_DRM_H__
 #define __VSP1_DRM_H__
 
+#include <linux/videodev2.h>
+
 #include "vsp1_pipe.h"
 
 /**
  * vsp1_drm - State for the API exposed to the DRM driver
  * @pipe: the VSP1 pipeline used for display
  * @num_inputs: number of active pipeline inputs at the beginning of an update
- * @update: the pipeline configuration has been updated
+ * @planes: source crop rectangle, destination compose rectangle and z-order
+ *	position for every input
  */
 struct vsp1_drm {
 	struct vsp1_pipeline pipe;
 	unsigned int num_inputs;
-	bool update;
+	struct {
+		bool enabled;
+		struct v4l2_rect crop;
+		struct v4l2_rect compose;
+		unsigned int zpos;
+	} inputs[VSP1_MAX_RPF];
 };
 
 int vsp1_drm_init(struct vsp1_device *vsp1);
diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 4913b933562c..15e028321fa1 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -200,6 +200,7 @@ void vsp1_pipeline_init(struct vsp1_pipeline *pipe)
 	pipe->state = VSP1_PIPELINE_STOPPED;
 }
 
+/* Must be called with the pipe irqlock held. */
 void vsp1_pipeline_run(struct vsp1_pipeline *pipe)
 {
 	struct vsp1_device *vsp1 = pipe->output->entity.vsp1;
diff --git a/include/media/vsp1.h b/include/media/vsp1.h
index d01f7cb8f691..e54a493bd3ff 100644
--- a/include/media/vsp1.h
+++ b/include/media/vsp1.h
@@ -24,10 +24,20 @@ int vsp1_du_setup_lif(struct device *dev, unsigned int width,
 		      unsigned int height);
 
 void vsp1_du_atomic_begin(struct device *dev);
-int vsp1_du_atomic_update(struct device *dev, unsigned int rpf, u32 pixelformat,
-			  unsigned int pitch, dma_addr_t mem[2],
-			  const struct v4l2_rect *src,
-			  const struct v4l2_rect *dst);
+int vsp1_du_atomic_update_ext(struct device *dev, unsigned int rpf,
+			      u32 pixelformat, unsigned int pitch,
+			      dma_addr_t mem[2], const struct v4l2_rect *src,
+			      const struct v4l2_rect *dst, unsigned int zpos);
 void vsp1_du_atomic_flush(struct device *dev);
 
+static inline int vsp1_du_atomic_update(struct device *dev,
+					unsigned int rpf_index, u32 pixelformat,
+					unsigned int pitch, dma_addr_t mem[2],
+					const struct v4l2_rect *src,
+					const struct v4l2_rect *dst)
+{
+	return vsp1_du_atomic_update_ext(dev, rpf_index, pixelformat, pitch,
+					 mem, src, dst, 0);
+}
+
 #endif /* __MEDIA_VSP1_H__ */
-- 
2.7.3

[PATCH v2 52/54] v4l: vsp1: Add global alpha support for DRM pipeline

[Laurent Pinchart]

Make the global alpha multiplier of DRM planes configurable. All the
necessary infrastructure is there, we just need to store the alpha value
passed through the DRM API.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_drm.c | 5 ++++-
 include/media/vsp1.h                   | 5 +++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_drm.c b/drivers/media/platform/vsp1/vsp1_drm.c
index d85cb0e258c9..fc4bbc401e67 100644
--- a/drivers/media/platform/vsp1/vsp1_drm.c
+++ b/drivers/media/platform/vsp1/vsp1_drm.c
@@ -235,6 +235,7 @@ EXPORT_SYMBOL_GPL(vsp1_du_atomic_begin);
  * @mem: DMA addresses of the memory buffers (one per plane)
  * @src: the source crop rectangle for the RPF
  * @dst: the destination compose rectangle for the BRU input
+ * @alpha: global alpha value for the input
  * @zpos: the Z-order position of the input
  *
  * Configure the VSP to perform composition of the image referenced by @mem
@@ -263,7 +264,8 @@ EXPORT_SYMBOL_GPL(vsp1_du_atomic_begin);
 int vsp1_du_atomic_update_ext(struct device *dev, unsigned int rpf_index,
 			      u32 pixelformat, unsigned int pitch,
 			      dma_addr_t mem[2], const struct v4l2_rect *src,
-			      const struct v4l2_rect *dst, unsigned int zpos)
+			      const struct v4l2_rect *dst, unsigned int alpha,
+			      unsigned int zpos)
 {
 	struct vsp1_device *vsp1 = dev_get_drvdata(dev);
 	const struct vsp1_format_info *fmtinfo;
@@ -303,6 +305,7 @@ int vsp1_du_atomic_update_ext(struct device *dev, unsigned int rpf_index,
 	rpf->format.num_planes = fmtinfo->planes;
 	rpf->format.plane_fmt[0].bytesperline = pitch;
 	rpf->format.plane_fmt[1].bytesperline = pitch;
+	rpf->alpha = alpha;
 
 	rpf->mem.addr[0] = mem[0];
 	rpf->mem.addr[1] = mem[1];
diff --git a/include/media/vsp1.h b/include/media/vsp1.h
index e54a493bd3ff..3e654a0455bd 100644
--- a/include/media/vsp1.h
+++ b/include/media/vsp1.h
@@ -27,7 +27,8 @@ void vsp1_du_atomic_begin(struct device *dev);
 int vsp1_du_atomic_update_ext(struct device *dev, unsigned int rpf,
 			      u32 pixelformat, unsigned int pitch,
 			      dma_addr_t mem[2], const struct v4l2_rect *src,
-			      const struct v4l2_rect *dst, unsigned int zpos);
+			      const struct v4l2_rect *dst, unsigned int alpha,
+			      unsigned int zpos);
 void vsp1_du_atomic_flush(struct device *dev);
 
 static inline int vsp1_du_atomic_update(struct device *dev,
@@ -37,7 +38,7 @@ static inline int vsp1_du_atomic_update(struct device *dev,
 					const struct v4l2_rect *dst)
 {
 	return vsp1_du_atomic_update_ext(dev, rpf_index, pixelformat, pitch,
-					 mem, src, dst, 0);
+					 mem, src, dst, 255, 0);
 }
 
 #endif /* __MEDIA_VSP1_H__ */
-- 
2.7.3

[PATCH v2 53/54] v4l: vsp1: Fix V4L2_PIX_FMT_XRGB444 format definition

[Laurent Pinchart]

The format is erroneously defined with an alpha channel. Fix it.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_pipe.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/vsp1/vsp1_pipe.c b/drivers/media/platform/vsp1/vsp1_pipe.c
index 15e028321fa1..4f3b4a1d028a 100644
--- a/drivers/media/platform/vsp1/vsp1_pipe.c
+++ b/drivers/media/platform/vsp1/vsp1_pipe.c
@@ -43,7 +43,7 @@ static const struct vsp1_format_info vsp1_video_formats[] = {
 	{ V4L2_PIX_FMT_XRGB444, MEDIA_BUS_FMT_ARGB8888_1X32,
 	  VI6_FMT_XRGB_4444, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
 	  VI6_RPF_DSWAP_P_WDS,
-	  1, { 16, 0, 0 }, false, false, 1, 1, true },
+	  1, { 16, 0, 0 }, false, false, 1, 1, false },
 	{ V4L2_PIX_FMT_ARGB555, MEDIA_BUS_FMT_ARGB8888_1X32,
 	  VI6_FMT_ARGB_1555, VI6_RPF_DSWAP_P_LLS | VI6_RPF_DSWAP_P_LWS |
 	  VI6_RPF_DSWAP_P_WDS,
-- 
2.7.3

[PATCH v2 54/54] v4l: vsp1: Update WPF and LIF maximum sizes for Gen3

[Laurent Pinchart]

The maximum image size supported by the WPF is 2048x2048 on Gen2 and
8190x8190 on Gen3. Update the code accordingly, and fix the maximum LIF
size for both Gen2 and Gen3.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/media/platform/vsp1/vsp1_lif.c |  2 +-
 drivers/media/platform/vsp1/vsp1_wpf.c | 15 +++++++++++----
 2 files changed, 12 insertions(+), 5 deletions(-)

diff --git a/drivers/media/platform/vsp1/vsp1_lif.c b/drivers/media/platform/vsp1/vsp1_lif.c
index d1d52a25c15b..4275ec3d2043 100644
--- a/drivers/media/platform/vsp1/vsp1_lif.c
+++ b/drivers/media/platform/vsp1/vsp1_lif.c
@@ -21,7 +21,7 @@
 #include "vsp1_lif.h"
 
 #define LIF_MIN_SIZE				2U
-#define LIF_MAX_SIZE				2048U
+#define LIF_MAX_SIZE				8190U
 
 /* -----------------------------------------------------------------------------
  * Device Access
diff --git a/drivers/media/platform/vsp1/vsp1_wpf.c b/drivers/media/platform/vsp1/vsp1_wpf.c
index a88ed0fc69ac..a50624477d5f 100644
--- a/drivers/media/platform/vsp1/vsp1_wpf.c
+++ b/drivers/media/platform/vsp1/vsp1_wpf.c
@@ -21,8 +21,10 @@
 #include "vsp1_rwpf.h"
 #include "vsp1_video.h"
 
-#define WPF_MAX_WIDTH				2048
-#define WPF_MAX_HEIGHT				2048
+#define WPF_GEN2_MAX_WIDTH			2048U
+#define WPF_GEN2_MAX_HEIGHT			2048U
+#define WPF_GEN3_MAX_WIDTH			8190U
+#define WPF_GEN3_MAX_HEIGHT			8190U
 
 /* -----------------------------------------------------------------------------
  * Device Access
@@ -201,8 +203,13 @@ struct vsp1_rwpf *vsp1_wpf_create(struct vsp1_device *vsp1, unsigned int index)
 	if (wpf == NULL)
 		return ERR_PTR(-ENOMEM);
 
-	wpf->max_width = WPF_MAX_WIDTH;
-	wpf->max_height = WPF_MAX_HEIGHT;
+	if (vsp1->info->gen == 2) {
+		wpf->max_width = WPF_GEN2_MAX_WIDTH;
+		wpf->max_height = WPF_GEN2_MAX_HEIGHT;
+	} else {
+		wpf->max_width = WPF_GEN3_MAX_WIDTH;
+		wpf->max_height = WPF_GEN3_MAX_HEIGHT;
+	}
 
 	wpf->entity.ops = &wpf_entity_ops;
 	wpf->entity.type = VSP1_ENTITY_WPF;
-- 
2.7.3

Re: [PATCH v2 03/54] v4l: subdev: Call pad init_cfg operation when opening subdevs

[Hans Verkuil]

On 03/25/2016 11:43 AM, Laurent Pinchart wrote:
> The subdev core code currently rely on the subdev open handler to
> initialize the file handle's pad configuration, even though subdevs now
> have a pad operation dedicated for that purpose.
> 
> As a first step towards migration to init_cfg, call the operation
> operation in the subdev core open implementation. Subdevs that haven't
> been moved to init_cfg yet will just continue implementing pad config
> initialization in their open handler.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index d4007f8f58d1..1fa6b713ee19 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -83,6 +83,12 @@ static int subdev_open(struct file *file)
>  	}
>  #endif
>  
> +#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
> +	ret = v4l2_subdev_call(sd, pad, init_cfg, subdev_fh->pad);
> +	if (ret < 0 && ret != -ENOIOCTLCMD)
> +		goto err;
> +#endif

Am I missing something here? Doesn't the subdev_fh_init() call earlier in this
function call pad.init_cfg already?

Regards,

	Hans

> +
>  	if (sd->internal_ops && sd->internal_ops->open) {
>  		ret = sd->internal_ops->open(sd, subdev_fh);
>  		if (ret < 0)
> 

Re: [PATCH v2 01/54] media: Add video processing entity functions

[Hans Verkuil]

On 03/25/2016 11:43 AM, Laurent Pinchart wrote:
> Add composer, format converter and scaler functions, as well as generic
> video processing to be used when no other processing function is
> applicable.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  Documentation/DocBook/media/v4l/media-types.xml | 34 +++++++++++++++++++++++++
>  include/uapi/linux/media.h                      |  8 ++++++
>  2 files changed, 42 insertions(+)
> 
> diff --git a/Documentation/DocBook/media/v4l/media-types.xml b/Documentation/DocBook/media/v4l/media-types.xml
> index 5e3f20fdcf17..a6e171e80bce 100644
> --- a/Documentation/DocBook/media/v4l/media-types.xml
> +++ b/Documentation/DocBook/media/v4l/media-types.xml
> @@ -121,6 +121,40 @@
>  	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
>  	    <entry>Audio Mixer Function Entity.</entry>
>  	  </row>
> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
> +	    <entry>Generic video processing, when no other processing function
> +		   is applicable.
> +	    </entry>

Does someone in this patch series use this one? If not, then just drop it. And
if there is one, which patch is it?

> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
> +	    <entry>Video composer (blender). An entity capable of video
> +		   composing must have at least two sink pads and one source
> +		   pad, and composes input video frames onto output video
> +		   frames. Composition can be performed using alpha blending,
> +		   color keying, raster operations (ROP), stitching or any other
> +		   mean.

s/mean/means/

> +	    </entry>
> +	  </row>
> +	  </row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
> +	    <entry>Video format converter. An entity capable of video format
> +		   conversion must have at least one sink pad and one source
> +		   pad, and convert the format of pixels received on its sink
> +		   pad(s) to a different format output on its source pad(s).
> +	    </entry>

Does this cover a de-interlacer?

> +	  </row>
> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
> +	    <entry>Video scaler. An entity capable of video scaling must have
> +		   at least one sink pad and one source pad, and scaling the
> +		   video frame(s) received on its sink pad(s) to a different
> +		   resolution output on its source pad(s). The range of
> +		   supported scaling ratios is entity-specific and can differ
> +		   between the horizontal and vertical directions. In particular
> +		   scaling can be supported in one direction only.
> +	    </entry>
> +	  </row>
>  	</tbody>
>        </tgroup>
>      </table>

Regards,

	Hans

> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index df59edee25d1..884ec1cae09d 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -95,6 +95,14 @@ struct media_device_info {
>  #define MEDIA_ENT_F_AUDIO_MIXER		(MEDIA_ENT_F_BASE + 0x03003)
>  
>  /*
> + * Processing entities
> + */
> +#define MEDIA_ENT_F_PROC_VIDEO_GENERIC		(MEDIA_ENT_F_BASE + 0x4001)
> +#define MEDIA_ENT_F_PROC_VIDEO_COMPOSER		(MEDIA_ENT_F_BASE + 0x4002)
> +#define MEDIA_ENT_F_PROC_VIDEO_CONVERTER	(MEDIA_ENT_F_BASE + 0x4003)
> +#define MEDIA_ENT_F_PROC_VIDEO_SCALER		(MEDIA_ENT_F_BASE + 0x4004)
> +
> +/*
>   * Connectors
>   */
>  /* It is a responsibility of the entity drivers to add connectors and links */
> 

Re: [PATCH v2 01/54] media: Add video processing entity functions

[Laurent Pinchart]

Hi Hans,

On Friday 25 Mar 2016 13:44:27 Hans Verkuil wrote:
> On 03/25/2016 11:43 AM, Laurent Pinchart wrote:
> > Add composer, format converter and scaler functions, as well as generic
> > video processing to be used when no other processing function is
> > applicable.
> > 
> > Signed-off-by: Laurent Pinchart
> > <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> > 
> >  Documentation/DocBook/media/v4l/media-types.xml | 34 ++++++++++++++++++++
> >  include/uapi/linux/media.h                      |  8 ++++++
> >  2 files changed, 42 insertions(+)
> > 
> > diff --git a/Documentation/DocBook/media/v4l/media-types.xml
> > b/Documentation/DocBook/media/v4l/media-types.xml index
> > 5e3f20fdcf17..a6e171e80bce 100644
> > --- a/Documentation/DocBook/media/v4l/media-types.xml
> > +++ b/Documentation/DocBook/media/v4l/media-types.xml
> > @@ -121,6 +121,40 @@
> > 
> >  	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
> >  	    <entry>Audio Mixer Function Entity.</entry>
> >  	  
> >  	  </row>
> > 
> > +	  <row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
> > +	    <entry>Generic video processing, when no other processing 
function
> > +		   is applicable.
> > +	    </entry>
> 
> Does someone in this patch series use this one? If not, then just drop it.
> And if there is one, which patch is it?

Yes, I have three entities using this, two look up tables (1D and 3D) and an 
entity handling the interface between the VSP and the display device.

> > +	  <row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
> > +	    <entry>Video composer (blender). An entity capable of video
> > +		   composing must have at least two sink pads and one source
> > +		   pad, and composes input video frames onto output video
> > +		   frames. Composition can be performed using alpha blending,
> > +		   color keying, raster operations (ROP), stitching or any other
> > +		   mean.
> 
> s/mean/means/

Will fix, thanks.

> > +	    </entry>
> > +	  </row>
> > +	  </row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
> > +	    <entry>Video format converter. An entity capable of video format
> > +		   conversion must have at least one sink pad and one source
> > +		   pad, and convert the format of pixels received on its sink
> > +		   pad(s) to a different format output on its source pad(s).
> > +	    </entry>
> 
> Does this cover a de-interlacer?

Deinterlacing and pixel format conversion seem to be different concepts to me, 
I wouldn't include deinterlacers here.

> > +	  </row>
> > +	  <row>
> > +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
> > +	    <entry>Video scaler. An entity capable of video scaling must have
> > +		   at least one sink pad and one source pad, and scaling the
> > +		   video frame(s) received on its sink pad(s) to a different
> > +		   resolution output on its source pad(s). The range of
> > +		   supported scaling ratios is entity-specific and can differ
> > +		   between the horizontal and vertical directions. In particular
> > +		   scaling can be supported in one direction only.
> > +	    </entry>
> > +	  </row>
> > 
> >  	</tbody>
> >  	
> >        </tgroup>
> >      
> >      </table>
> 
> Regards,
> 
> 	Hans
> 
> > diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> > index df59edee25d1..884ec1cae09d 100644
> > --- a/include/uapi/linux/media.h
> > +++ b/include/uapi/linux/media.h
> > @@ -95,6 +95,14 @@ struct media_device_info {
> > 
> >  #define MEDIA_ENT_F_AUDIO_MIXER		(MEDIA_ENT_F_BASE + 0x03003)
> >  
> >  /*
> > 
> > + * Processing entities
> > + */
> > +#define MEDIA_ENT_F_PROC_VIDEO_GENERIC		(MEDIA_ENT_F_BASE + 0x4001)
> > +#define MEDIA_ENT_F_PROC_VIDEO_COMPOSER		(MEDIA_ENT_F_BASE + 0x4002)
> > +#define MEDIA_ENT_F_PROC_VIDEO_CONVERTER	(MEDIA_ENT_F_BASE + 0x4003)
> > +#define MEDIA_ENT_F_PROC_VIDEO_SCALER		(MEDIA_ENT_F_BASE + 0x4004)
> > +
> > +/*
> > 
> >   * Connectors
> >   */
> >  
> >  /* It is a responsibility of the entity drivers to add connectors and
> >  links */

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 03/54] v4l: subdev: Call pad init_cfg operation when opening subdevs

[Laurent Pinchart]

Hi Hans,

On Friday 25 Mar 2016 13:40:55 Hans Verkuil wrote:
> On 03/25/2016 11:43 AM, Laurent Pinchart wrote:
> > The subdev core code currently rely on the subdev open handler to
> > initialize the file handle's pad configuration, even though subdevs now
> > have a pad operation dedicated for that purpose.
> > 
> > As a first step towards migration to init_cfg, call the operation
> > operation in the subdev core open implementation. Subdevs that haven't
> > been moved to init_cfg yet will just continue implementing pad config
> > initialization in their open handler.
> > 
> > Signed-off-by: Laurent Pinchart
> > <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> > 
> >  drivers/media/v4l2-core/v4l2-subdev.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c
> > b/drivers/media/v4l2-core/v4l2-subdev.c index d4007f8f58d1..1fa6b713ee19
> > 100644
> > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > @@ -83,6 +83,12 @@ static int subdev_open(struct file *file)
> > 
> >  	}
> >  
> >  #endif
> > 
> > +#if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
> > +	ret = v4l2_subdev_call(sd, pad, init_cfg, subdev_fh->pad);
> > +	if (ret < 0 && ret != -ENOIOCTLCMD)
> > +		goto err;
> > +#endif
> 
> Am I missing something here? Doesn't the subdev_fh_init() call earlier in
> this function call pad.init_cfg already?

You're absolutely right, I'll drop this patch.

> > +
> > 
> >  	if (sd->internal_ops && sd->internal_ops->open) {
> >  	
> >  		ret = sd->internal_ops->open(sd, subdev_fh);
> >  		if (ret < 0)

-- 
Regards,

Laurent Pinchart

[PATCH v3] media: Add video processing entity functions

[Laurent Pinchart]

Add composer, format converter and scaler functions, as well as generic
video processing to be used when no other processing function is
applicable.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 Documentation/DocBook/media/v4l/media-types.xml | 34 +++++++++++++++++++++++++
 include/uapi/linux/media.h                      |  8 ++++++
 2 files changed, 42 insertions(+)

Changes since v2:

- Fix typo (any other mean -> any other means)

diff --git a/Documentation/DocBook/media/v4l/media-types.xml b/Documentation/DocBook/media/v4l/media-types.xml
index 5e3f20fdcf17..38e8d6c25d49 100644
--- a/Documentation/DocBook/media/v4l/media-types.xml
+++ b/Documentation/DocBook/media/v4l/media-types.xml
@@ -121,6 +121,40 @@
 	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
 	    <entry>Audio Mixer Function Entity.</entry>
 	  </row>
+	  <row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
+	    <entry>Generic video processing, when no other processing function
+		   is applicable.
+	    </entry>
+	  <row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
+	    <entry>Video composer (blender). An entity capable of video
+		   composing must have at least two sink pads and one source
+		   pad, and composes input video frames onto output video
+		   frames. Composition can be performed using alpha blending,
+		   color keying, raster operations (ROP), stitching or any other
+		   means.
+	    </entry>
+	  </row>
+	  </row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
+	    <entry>Video format converter. An entity capable of video format
+		   conversion must have at least one sink pad and one source
+		   pad, and convert the format of pixels received on its sink
+		   pad(s) to a different format output on its source pad(s).
+	    </entry>
+	  </row>
+	  <row>
+	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
+	    <entry>Video scaler. An entity capable of video scaling must have
+		   at least one sink pad and one source pad, and scaling the
+		   video frame(s) received on its sink pad(s) to a different
+		   resolution output on its source pad(s). The range of
+		   supported scaling ratios is entity-specific and can differ
+		   between the horizontal and vertical directions. In particular
+		   scaling can be supported in one direction only.
+	    </entry>
+	  </row>
 	</tbody>
       </tgroup>
     </table>
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
index df59edee25d1..884ec1cae09d 100644
--- a/include/uapi/linux/media.h
+++ b/include/uapi/linux/media.h
@@ -95,6 +95,14 @@ struct media_device_info {
 #define MEDIA_ENT_F_AUDIO_MIXER		(MEDIA_ENT_F_BASE + 0x03003)
 
 /*
+ * Processing entities
+ */
+#define MEDIA_ENT_F_PROC_VIDEO_GENERIC		(MEDIA_ENT_F_BASE + 0x4001)
+#define MEDIA_ENT_F_PROC_VIDEO_COMPOSER		(MEDIA_ENT_F_BASE + 0x4002)
+#define MEDIA_ENT_F_PROC_VIDEO_CONVERTER	(MEDIA_ENT_F_BASE + 0x4003)
+#define MEDIA_ENT_F_PROC_VIDEO_SCALER		(MEDIA_ENT_F_BASE + 0x4004)
+
+/*
  * Connectors
  */
 /* It is a responsibility of the entity drivers to add connectors and links */
-- 
2.7.3

Re: [PATCH v3] media: Add video processing entity functions

[Hans Verkuil]

On 03/25/2016 03:19 PM, Laurent Pinchart wrote:
> Add composer, format converter and scaler functions, as well as generic
> video processing to be used when no other processing function is
> applicable.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  Documentation/DocBook/media/v4l/media-types.xml | 34 +++++++++++++++++++++++++
>  include/uapi/linux/media.h                      |  8 ++++++
>  2 files changed, 42 insertions(+)
> 
> Changes since v2:
> 
> - Fix typo (any other mean -> any other means)
> 
> diff --git a/Documentation/DocBook/media/v4l/media-types.xml b/Documentation/DocBook/media/v4l/media-types.xml
> index 5e3f20fdcf17..38e8d6c25d49 100644
> --- a/Documentation/DocBook/media/v4l/media-types.xml
> +++ b/Documentation/DocBook/media/v4l/media-types.xml
> @@ -121,6 +121,40 @@
>  	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
>  	    <entry>Audio Mixer Function Entity.</entry>
>  	  </row>
> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
> +	    <entry>Generic video processing, when no other processing function
> +		   is applicable.
> +	    </entry>

I'll be honest and say that I don't really like this. A VIDEO_GENERIC function
is only marginally more useful than UNKNOWN. And I think I prefer UNKNOWN for
now until we have a clear picture how these functions are going to work. The
main missing piece in this puzzle are properties that allow us to register
multiple functions and some decision as to what the scope of 'functions' is going
to be.

You mentioned that you have a few entities that are using this function, but
if you would specify the exact function of those entities, what would the
function name(s) be?

> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
> +	    <entry>Video composer (blender). An entity capable of video
> +		   composing must have at least two sink pads and one source
> +		   pad, and composes input video frames onto output video
> +		   frames. Composition can be performed using alpha blending,
> +		   color keying, raster operations (ROP), stitching or any other
> +		   means.
> +	    </entry>
> +	  </row>

This one looks OK to me.

> +	  </row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
> +	    <entry>Video format converter. An entity capable of video format
> +		   conversion must have at least one sink pad and one source
> +		   pad, and convert the format of pixels received on its sink
> +		   pad(s) to a different format output on its source pad(s).
> +	    </entry>
> +	  </row>

This is too vague as well, I think. You said that you don't consider de-interlacing
a converter function, but what about colorimetry conversion? Debayer? 4:2:2 to 4:2:0
conversion or vice versa?

> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
> +	    <entry>Video scaler. An entity capable of video scaling must have
> +		   at least one sink pad and one source pad, and scaling the
> +		   video frame(s) received on its sink pad(s) to a different
> +		   resolution output on its source pad(s). The range of
> +		   supported scaling ratios is entity-specific and can differ
> +		   between the horizontal and vertical directions. In particular
> +		   scaling can be supported in one direction only.
> +	    </entry>
> +	  </row>

This looks OK too, although would sensor binning and/or skipping also be considered
scaling?

Regards,

	Hans

>  	</tbody>
>        </tgroup>
>      </table>
> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index df59edee25d1..884ec1cae09d 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -95,6 +95,14 @@ struct media_device_info {
>  #define MEDIA_ENT_F_AUDIO_MIXER		(MEDIA_ENT_F_BASE + 0x03003)
>  
>  /*
> + * Processing entities
> + */
> +#define MEDIA_ENT_F_PROC_VIDEO_GENERIC		(MEDIA_ENT_F_BASE + 0x4001)
> +#define MEDIA_ENT_F_PROC_VIDEO_COMPOSER		(MEDIA_ENT_F_BASE + 0x4002)
> +#define MEDIA_ENT_F_PROC_VIDEO_CONVERTER	(MEDIA_ENT_F_BASE + 0x4003)
> +#define MEDIA_ENT_F_PROC_VIDEO_SCALER		(MEDIA_ENT_F_BASE + 0x4004)
> +
> +/*
>   * Connectors
>   */
>  /* It is a responsibility of the entity drivers to add connectors and links */
> 

Re: [PATCH v3] media: Add video processing entity functions

[Laurent Pinchart]

Hi Hans,

On Friday 25 Mar 2016 16:14:59 Hans Verkuil wrote:
> On 03/25/2016 03:19 PM, Laurent Pinchart wrote:
> > Add composer, format converter and scaler functions, as well as generic
> > video processing to be used when no other processing function is
> > applicable.
> > 
> > Signed-off-by: Laurent Pinchart
> > <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> > 
> >  Documentation/DocBook/media/v4l/media-types.xml | 34 ++++++++++++++++++++
> >  include/uapi/linux/media.h                      |  8 ++++++
> >  2 files changed, 42 insertions(+)
> > 
> > Changes since v2:
> > 
> > - Fix typo (any other mean -> any other means)
> > 
> > diff --git a/Documentation/DocBook/media/v4l/media-types.xml
> > b/Documentation/DocBook/media/v4l/media-types.xml index
> > 5e3f20fdcf17..38e8d6c25d49 100644
> > --- a/Documentation/DocBook/media/v4l/media-types.xml
> > +++ b/Documentation/DocBook/media/v4l/media-types.xml
> > @@ -121,6 +121,40 @@
> >  	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
> >  	    <entry>Audio Mixer Function Entity.</entry>
> >  	  </row>
> > +	  <row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
> > +	    <entry>Generic video processing, when no other processing 
function
> > +		   is applicable.
> > +	    </entry>
> 
> I'll be honest and say that I don't really like this. A VIDEO_GENERIC
> function is only marginally more useful than UNKNOWN. And I think I prefer
> UNKNOWN for now until we have a clear picture how these functions are going
> to work. The main missing piece in this puzzle are properties that allow us
> to register multiple functions and some decision as to what the scope of
> 'functions' is going to be.
> 
> You mentioned that you have a few entities that are using this function, but
> if you would specify the exact function of those entities, what would the
> function name(s) be?

I'm not sure, otherwise I would have proposed more precise functions :-)

Two of the entities just apply look up tables. They can thus be used for 
various purpose, such as gamma correction, A-law/µ-law (de)compression, ...

The last entity is an interface between the VSP and the display device and 
just handles buffering, clock domain crossing and synchronization.

> > +	  <row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
> > +	    <entry>Video composer (blender). An entity capable of video
> > +		   composing must have at least two sink pads and one source
> > +		   pad, and composes input video frames onto output video
> > +		   frames. Composition can be performed using alpha blending,
> > +		   color keying, raster operations (ROP), stitching or any other
> > +		   means.
> > +	    </entry>
> > +	  </row>
> 
> This one looks OK to me.
> 
> > +	  </row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
> > +	    <entry>Video format converter. An entity capable of video format
> > +		   conversion must have at least one sink pad and one source
> > +		   pad, and convert the format of pixels received on its sink
> > +		   pad(s) to a different format output on its source pad(s).
> > +	    </entry>
> > +	  </row>
> 
> This is too vague as well, I think. You said that you don't consider
> de-interlacing a converter function, but what about colorimetry conversion?
> Debayer? 4:2:2 to 4:2:0 conversion or vice versa?

I'd consider that as video format conversion, yes.

The three entities that implement this function in the vsp1 driver are 
ARGB8888 <-> AHSV8888 converters and RGB <-> YUV converters (with various RGB 
and YUV formats supported).

> > +	  <row>
> > +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
> > +	    <entry>Video scaler. An entity capable of video scaling must have
> > +		   at least one sink pad and one source pad, and scaling the
> > +		   video frame(s) received on its sink pad(s) to a different
> > +		   resolution output on its source pad(s). The range of
> > +		   supported scaling ratios is entity-specific and can differ
> > +		   between the horizontal and vertical directions. In particular
> > +		   scaling can be supported in one direction only.
> > +	    </entry>
> > +	  </row>
> 
> This looks OK too, although would sensor binning and/or skipping also be
> considered scaling?

I would consider them as scaling, yes. Sakari, any opinion on that ?


> >  	</tbody>
> >        </tgroup>
> >      </table>

-- 
Regards,

Laurent Pinchart

[PATCH v3] v4l: subdev: Add pad config allocator and init

[Laurent Pinchart]

From: Laurent Pinchart <laurent.pinchart@linaro.org>

Add a new subdev operation to initialize a subdev pad config array, and
a helper function to allocate and initialize the array. This can be used
by bridge drivers to implement try format based on subdev pad
operations.

Signed-off-by: Laurent Pinchart <laurent.pinchart@linaro.org>
Acked-by: Vaibhav Hiremath <vaibhav.hiremath@linaro.org>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
---
 drivers/media/v4l2-core/v4l2-subdev.c | 37 ++++++++++++++++++++++++++++++++---
 include/media/v4l2-subdev.h           |  8 ++++++++
 2 files changed, 42 insertions(+), 3 deletions(-)

Changes since v2:

- Don't fail due to pad config allocation when the subdev has no pad.

diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
index 0fa60801a428..8c6f016d1791 100644
--- a/drivers/media/v4l2-core/v4l2-subdev.c
+++ b/drivers/media/v4l2-core/v4l2-subdev.c
@@ -35,9 +35,11 @@
 static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
 {
 #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
-	fh->pad = kzalloc(sizeof(*fh->pad) * sd->entity.num_pads, GFP_KERNEL);
-	if (fh->pad == NULL)
-		return -ENOMEM;
+	if (sd->entity.num_pads) {
+		fh->pad = v4l2_subdev_alloc_pad_config(sd);
+		if (fh->pad == NULL)
+			return -ENOMEM;
+	}
 #endif
 	return 0;
 }
@@ -569,6 +571,35 @@ int v4l2_subdev_link_validate(struct media_link *link)
 		sink, link, &source_fmt, &sink_fmt);
 }
 EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
+
+struct v4l2_subdev_pad_config *
+v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd)
+{
+	struct v4l2_subdev_pad_config *cfg;
+	int ret;
+
+	if (!sd->entity.num_pads)
+		return NULL;
+
+	cfg = kcalloc(sd->entity.num_pads, sizeof(*cfg), GFP_KERNEL);
+	if (!cfg)
+		return NULL;
+
+	ret = v4l2_subdev_call(sd, pad, init_cfg, cfg);
+	if (ret < 0 && ret != -ENOIOCTLCMD) {
+		kfree(cfg);
+		return NULL;
+	}
+
+	return cfg;
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_alloc_pad_config);
+
+void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg)
+{
+	kfree(cfg);
+}
+EXPORT_SYMBOL_GPL(v4l2_subdev_free_pad_config);
 #endif /* CONFIG_MEDIA_CONTROLLER */
 
 void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
index 11e2dfec0198..32fc7a4beb5e 100644
--- a/include/media/v4l2-subdev.h
+++ b/include/media/v4l2-subdev.h
@@ -572,6 +572,7 @@ struct v4l2_subdev_pad_config {
 /**
  * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
  *
+ * @init_cfg: initialize the pad config to default values
  * @enum_mbus_code: callback for VIDIOC_SUBDEV_ENUM_MBUS_CODE ioctl handler
  *		    code.
  * @enum_frame_size: callback for VIDIOC_SUBDEV_ENUM_FRAME_SIZE ioctl handler
@@ -607,6 +608,8 @@ struct v4l2_subdev_pad_config {
  *                  may be adjusted by the subdev driver to device capabilities.
  */
 struct v4l2_subdev_pad_ops {
+	int (*init_cfg)(struct v4l2_subdev *sd,
+			struct v4l2_subdev_pad_config *cfg);
 	int (*enum_mbus_code)(struct v4l2_subdev *sd,
 			      struct v4l2_subdev_pad_config *cfg,
 			      struct v4l2_subdev_mbus_code_enum *code);
@@ -801,7 +804,12 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
 				      struct v4l2_subdev_format *source_fmt,
 				      struct v4l2_subdev_format *sink_fmt);
 int v4l2_subdev_link_validate(struct media_link *link);
+
+struct v4l2_subdev_pad_config *
+v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd);
+void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg);
 #endif /* CONFIG_MEDIA_CONTROLLER */
+
 void v4l2_subdev_init(struct v4l2_subdev *sd,
 		      const struct v4l2_subdev_ops *ops);
 
-- 
2.7.3

Re: [PATCH v2 01/54] media: Add video processing entity functions

[Sakari Ailus]

Hi Laurent,

On Fri, Mar 25, 2016 at 12:43:35PM +0200, Laurent Pinchart wrote:
> Add composer, format converter and scaler functions, as well as generic
> video processing to be used when no other processing function is
> applicable.

How are these intended to be used?

Say, if a sub-device implements functionality that matches more than one of
these, do you pick one?

Supposedly you control at least some of this functionality using the
selections API, and frankly, I think the way it's currently defined in the
spec worked okay-ish for the devices at hand at the time, but defining that
the order of processing from the sink towards the source is sink crop, sink
compose and then source crop is not generic. We should have a better way to
tell this, using a similar API which is used to control the functionality,
just as is done with V4L2 controls.

> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>  Documentation/DocBook/media/v4l/media-types.xml | 34 +++++++++++++++++++++++++
>  include/uapi/linux/media.h                      |  8 ++++++
>  2 files changed, 42 insertions(+)
> 
> diff --git a/Documentation/DocBook/media/v4l/media-types.xml b/Documentation/DocBook/media/v4l/media-types.xml
> index 5e3f20fdcf17..a6e171e80bce 100644
> --- a/Documentation/DocBook/media/v4l/media-types.xml
> +++ b/Documentation/DocBook/media/v4l/media-types.xml
> @@ -121,6 +121,40 @@
>  	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
>  	    <entry>Audio Mixer Function Entity.</entry>
>  	  </row>
> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
> +	    <entry>Generic video processing, when no other processing function
> +		   is applicable.
> +	    </entry>
> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
> +	    <entry>Video composer (blender). An entity capable of video
> +		   composing must have at least two sink pads and one source
> +		   pad, and composes input video frames onto output video
> +		   frames. Composition can be performed using alpha blending,
> +		   color keying, raster operations (ROP), stitching or any other
> +		   mean.
> +	    </entry>
> +	  </row>
> +	  </row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
> +	    <entry>Video format converter. An entity capable of video format
> +		   conversion must have at least one sink pad and one source
> +		   pad, and convert the format of pixels received on its sink
> +		   pad(s) to a different format output on its source pad(s).
> +	    </entry>
> +	  </row>
> +	  <row>
> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
> +	    <entry>Video scaler. An entity capable of video scaling must have
> +		   at least one sink pad and one source pad, and scaling the
> +		   video frame(s) received on its sink pad(s) to a different
> +		   resolution output on its source pad(s). The range of
> +		   supported scaling ratios is entity-specific and can differ
> +		   between the horizontal and vertical directions. In particular
> +		   scaling can be supported in one direction only.
> +	    </entry>
> +	  </row>
>  	</tbody>
>        </tgroup>
>      </table>
> diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> index df59edee25d1..884ec1cae09d 100644
> --- a/include/uapi/linux/media.h
> +++ b/include/uapi/linux/media.h
> @@ -95,6 +95,14 @@ struct media_device_info {
>  #define MEDIA_ENT_F_AUDIO_MIXER		(MEDIA_ENT_F_BASE + 0x03003)
>  
>  /*
> + * Processing entities
> + */
> +#define MEDIA_ENT_F_PROC_VIDEO_GENERIC		(MEDIA_ENT_F_BASE + 0x4001)
> +#define MEDIA_ENT_F_PROC_VIDEO_COMPOSER		(MEDIA_ENT_F_BASE + 0x4002)
> +#define MEDIA_ENT_F_PROC_VIDEO_CONVERTER	(MEDIA_ENT_F_BASE + 0x4003)
> +#define MEDIA_ENT_F_PROC_VIDEO_SCALER		(MEDIA_ENT_F_BASE + 0x4004)
> +
> +/*
>   * Connectors
>   */
>  /* It is a responsibility of the entity drivers to add connectors and links */

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v3] v4l: subdev: Add pad config allocator and init

[Sakari Ailus]

Hi Laurent,

Thanks for the patch. One comment below...

On Fri, Mar 25, 2016 at 05:27:45PM +0200, Laurent Pinchart wrote:
> From: Laurent Pinchart <laurent.pinchart@linaro.org>
> 
> Add a new subdev operation to initialize a subdev pad config array, and
> a helper function to allocate and initialize the array. This can be used
> by bridge drivers to implement try format based on subdev pad
> operations.
> 
> Signed-off-by: Laurent Pinchart <laurent.pinchart@linaro.org>
> Acked-by: Vaibhav Hiremath <vaibhav.hiremath@linaro.org>
> Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
> ---
>  drivers/media/v4l2-core/v4l2-subdev.c | 37 ++++++++++++++++++++++++++++++++---
>  include/media/v4l2-subdev.h           |  8 ++++++++
>  2 files changed, 42 insertions(+), 3 deletions(-)
> 
> Changes since v2:
> 
> - Don't fail due to pad config allocation when the subdev has no pad.
> 
> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> index 0fa60801a428..8c6f016d1791 100644
> --- a/drivers/media/v4l2-core/v4l2-subdev.c
> +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> @@ -35,9 +35,11 @@
>  static int subdev_fh_init(struct v4l2_subdev_fh *fh, struct v4l2_subdev *sd)
>  {
>  #if defined(CONFIG_VIDEO_V4L2_SUBDEV_API)
> -	fh->pad = kzalloc(sizeof(*fh->pad) * sd->entity.num_pads, GFP_KERNEL);
> -	if (fh->pad == NULL)
> -		return -ENOMEM;
> +	if (sd->entity.num_pads) {
> +		fh->pad = v4l2_subdev_alloc_pad_config(sd);
> +		if (fh->pad == NULL)
> +			return -ENOMEM;
> +	}
>  #endif
>  	return 0;
>  }
> @@ -569,6 +571,35 @@ int v4l2_subdev_link_validate(struct media_link *link)
>  		sink, link, &source_fmt, &sink_fmt);
>  }
>  EXPORT_SYMBOL_GPL(v4l2_subdev_link_validate);
> +
> +struct v4l2_subdev_pad_config *
> +v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd)

What's the use case for calling v4l2_subdev_alloc_pad_config() elsewhere
than from subdev_fh_init()?

> +{
> +	struct v4l2_subdev_pad_config *cfg;
> +	int ret;
> +
> +	if (!sd->entity.num_pads)
> +		return NULL;
> +
> +	cfg = kcalloc(sd->entity.num_pads, sizeof(*cfg), GFP_KERNEL);
> +	if (!cfg)
> +		return NULL;
> +
> +	ret = v4l2_subdev_call(sd, pad, init_cfg, cfg);
> +	if (ret < 0 && ret != -ENOIOCTLCMD) {
> +		kfree(cfg);
> +		return NULL;
> +	}
> +
> +	return cfg;
> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_alloc_pad_config);
> +
> +void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg)
> +{
> +	kfree(cfg);

Not really a bug but --- I think it'd be cleaner to call this from
v4l2_fh_cleanup(), rather than freeing cfg directly using kfree().

> +}
> +EXPORT_SYMBOL_GPL(v4l2_subdev_free_pad_config);
>  #endif /* CONFIG_MEDIA_CONTROLLER */
>  
>  void v4l2_subdev_init(struct v4l2_subdev *sd, const struct v4l2_subdev_ops *ops)
> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> index 11e2dfec0198..32fc7a4beb5e 100644
> --- a/include/media/v4l2-subdev.h
> +++ b/include/media/v4l2-subdev.h
> @@ -572,6 +572,7 @@ struct v4l2_subdev_pad_config {
>  /**
>   * struct v4l2_subdev_pad_ops - v4l2-subdev pad level operations
>   *
> + * @init_cfg: initialize the pad config to default values
>   * @enum_mbus_code: callback for VIDIOC_SUBDEV_ENUM_MBUS_CODE ioctl handler
>   *		    code.
>   * @enum_frame_size: callback for VIDIOC_SUBDEV_ENUM_FRAME_SIZE ioctl handler
> @@ -607,6 +608,8 @@ struct v4l2_subdev_pad_config {
>   *                  may be adjusted by the subdev driver to device capabilities.
>   */
>  struct v4l2_subdev_pad_ops {
> +	int (*init_cfg)(struct v4l2_subdev *sd,
> +			struct v4l2_subdev_pad_config *cfg);
>  	int (*enum_mbus_code)(struct v4l2_subdev *sd,
>  			      struct v4l2_subdev_pad_config *cfg,
>  			      struct v4l2_subdev_mbus_code_enum *code);
> @@ -801,7 +804,12 @@ int v4l2_subdev_link_validate_default(struct v4l2_subdev *sd,
>  				      struct v4l2_subdev_format *source_fmt,
>  				      struct v4l2_subdev_format *sink_fmt);
>  int v4l2_subdev_link_validate(struct media_link *link);
> +
> +struct v4l2_subdev_pad_config *
> +v4l2_subdev_alloc_pad_config(struct v4l2_subdev *sd);
> +void v4l2_subdev_free_pad_config(struct v4l2_subdev_pad_config *cfg);
>  #endif /* CONFIG_MEDIA_CONTROLLER */
> +
>  void v4l2_subdev_init(struct v4l2_subdev *sd,
>  		      const struct v4l2_subdev_ops *ops);
>  

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v2 01/54] media: Add video processing entity functions

[Laurent Pinchart]

Hi Sakari,

On Tuesday 29 Mar 2016 02:01:55 Sakari Ailus wrote:
> Hi Laurent,
> 
> On Fri, Mar 25, 2016 at 12:43:35PM +0200, Laurent Pinchart wrote:
> > Add composer, format converter and scaler functions, as well as generic
> > video processing to be used when no other processing function is
> > applicable.
> 
> How are these intended to be used?
> 
> Say, if a sub-device implements functionality that matches more than one of
> these, do you pick one?

The whole point of functions is that they're not mutually exclusive, and the 
full list of functions will be reported as properties for the entity. The 
function field of the media entity structure stores the main function only.

> Supposedly you control at least some of this functionality using the
> selections API, and frankly, I think the way it's currently defined in the
> spec worked okay-ish for the devices at hand at the time, but defining that
> the order of processing from the sink towards the source is sink crop, sink
> compose and then source crop is not generic. We should have a better way to
> tell this, using a similar API which is used to control the functionality,
> just as is done with V4L2 controls.

Sure, but how is that related to this patch ? :-)

> > Signed-off-by: Laurent Pinchart
> > <laurent.pinchart+renesas@ideasonboard.com>
> > ---
> > 
> >  Documentation/DocBook/media/v4l/media-types.xml | 34 ++++++++++++++++++++
> >  include/uapi/linux/media.h                      |  8 ++++++
> >  2 files changed, 42 insertions(+)
> > 
> > diff --git a/Documentation/DocBook/media/v4l/media-types.xml
> > b/Documentation/DocBook/media/v4l/media-types.xml index
> > 5e3f20fdcf17..a6e171e80bce 100644
> > --- a/Documentation/DocBook/media/v4l/media-types.xml
> > +++ b/Documentation/DocBook/media/v4l/media-types.xml
> > @@ -121,6 +121,40 @@
> > 
> >  	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
> >  	    <entry>Audio Mixer Function Entity.</entry>
> >  	  
> >  	  </row>
> > 
> > +	  <row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
> > +	    <entry>Generic video processing, when no other processing 
function
> > +		   is applicable.
> > +	    </entry>
> > +	  <row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
> > +	    <entry>Video composer (blender). An entity capable of video
> > +		   composing must have at least two sink pads and one source
> > +		   pad, and composes input video frames onto output video
> > +		   frames. Composition can be performed using alpha blending,
> > +		   color keying, raster operations (ROP), stitching or any other
> > +		   mean.
> > +	    </entry>
> > +	  </row>
> > +	  </row>
> > +	    
<entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
> > +	    <entry>Video format converter. An entity capable of video format
> > +		   conversion must have at least one sink pad and one source
> > +		   pad, and convert the format of pixels received on its sink
> > +		   pad(s) to a different format output on its source pad(s).
> > +	    </entry>
> > +	  </row>
> > +	  <row>
> > +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
> > +	    <entry>Video scaler. An entity capable of video scaling must have
> > +		   at least one sink pad and one source pad, and scaling the
> > +		   video frame(s) received on its sink pad(s) to a different
> > +		   resolution output on its source pad(s). The range of
> > +		   supported scaling ratios is entity-specific and can differ
> > +		   between the horizontal and vertical directions. In particular
> > +		   scaling can be supported in one direction only.
> > +	    </entry>
> > +	  </row>
> > 
> >  	</tbody>
> >  	
> >        </tgroup>
> >      
> >      </table>
> > 
> > diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h
> > index df59edee25d1..884ec1cae09d 100644
> > --- a/include/uapi/linux/media.h
> > +++ b/include/uapi/linux/media.h
> > @@ -95,6 +95,14 @@ struct media_device_info {
> > 
> >  #define MEDIA_ENT_F_AUDIO_MIXER		(MEDIA_ENT_F_BASE + 0x03003)
> >  
> >  /*
> > 
> > + * Processing entities
> > + */
> > +#define MEDIA_ENT_F_PROC_VIDEO_GENERIC		(MEDIA_ENT_F_BASE + 0x4001)
> > +#define MEDIA_ENT_F_PROC_VIDEO_COMPOSER		(MEDIA_ENT_F_BASE + 0x4002)
> > +#define MEDIA_ENT_F_PROC_VIDEO_CONVERTER	(MEDIA_ENT_F_BASE + 0x4003)
> > +#define MEDIA_ENT_F_PROC_VIDEO_SCALER		(MEDIA_ENT_F_BASE + 0x4004)
> > +
> > +/*
> > 
> >   * Connectors
> >   */
> >  
> >  /* It is a responsibility of the entity drivers to add connectors and
> >  links */

-- 
Regards,

Laurent Pinchart

Re: [PATCH v2 01/54] media: Add video processing entity functions

[Laurent Pinchart]

Hi Sakari,

Do you still have issues with this patch ?

On Tuesday 29 Mar 2016 10:40:05 Laurent Pinchart wrote:
> On Tuesday 29 Mar 2016 02:01:55 Sakari Ailus wrote:
> > On Fri, Mar 25, 2016 at 12:43:35PM +0200, Laurent Pinchart wrote:
> > > Add composer, format converter and scaler functions, as well as generic
> > > video processing to be used when no other processing function is
> > > applicable.
> > 
> > How are these intended to be used?
> > 
> > Say, if a sub-device implements functionality that matches more than one
> > of these, do you pick one?
> 
> The whole point of functions is that they're not mutually exclusive, and the
> full list of functions will be reported as properties for the entity. The
> function field of the media entity structure stores the main function only.
>
> > Supposedly you control at least some of this functionality using the
> > selections API, and frankly, I think the way it's currently defined in the
> > spec worked okay-ish for the devices at hand at the time, but defining
> > that the order of processing from the sink towards the source is sink
> > crop, sink compose and then source crop is not generic. We should have a
> > better way to tell this, using a similar API which is used to control the
> > functionality, just as is done with V4L2 controls.
> 
> Sure, but how is that related to this patch ? :-)
> 
> > > Signed-off-by: Laurent Pinchart
> > > <laurent.pinchart+renesas@ideasonboard.com>
> > > ---
> > > 
> > >  Documentation/DocBook/media/v4l/media-types.xml | 34 ++++++++++++++++++
> > >  include/uapi/linux/media.h                      |  8 ++++++
> > >  2 files changed, 42 insertions(+)

-- 
Regards,

Laurent Pinchart

Re: [PATCH v3] media: Add video processing entity functions

[Laurent Pinchart]

Hi Hans and Sakari,

Ping ? I'd like to get the patch merged this week as part of a larger pull 
request.

On Friday 25 Mar 2016 17:23:25 Laurent Pinchart wrote:
> On Friday 25 Mar 2016 16:14:59 Hans Verkuil wrote:
> > On 03/25/2016 03:19 PM, Laurent Pinchart wrote:
> >> Add composer, format converter and scaler functions, as well as generic
> >> video processing to be used when no other processing function is
> >> applicable.
> >> 
> >> Signed-off-by: Laurent Pinchart
> >> <laurent.pinchart+renesas@ideasonboard.com>
> >> ---
> >> 
> >>  Documentation/DocBook/media/v4l/media-types.xml | 34 ++++++++++++++++++
> >>  include/uapi/linux/media.h                      |  8 ++++++
> >>  2 files changed, 42 insertions(+)
> >> 
> >> Changes since v2:
> >> 
> >> - Fix typo (any other mean -> any other means)
> >> 
> >> diff --git a/Documentation/DocBook/media/v4l/media-types.xml
> >> b/Documentation/DocBook/media/v4l/media-types.xml index
> >> 5e3f20fdcf17..38e8d6c25d49 100644
> >> --- a/Documentation/DocBook/media/v4l/media-types.xml
> >> +++ b/Documentation/DocBook/media/v4l/media-types.xml
> >> @@ -121,6 +121,40 @@
> >>  	    <entry><constant>MEDIA_ENT_F_AUDIO_MIXER</constant></entry>
> >>  	    <entry>Audio Mixer Function Entity.</entry>
> >>  	  </row>
> >> +	  <row>
> >> +
> >> <entry><constant>MEDIA_ENT_F_PROC_VIDEO_GENERIC</constant></entry>
> >> +	    <entry>Generic video processing, when no other processing
> >> function
> >> +		   is applicable.
> >> +	    </entry>
> > 
> > I'll be honest and say that I don't really like this. A VIDEO_GENERIC
> > function is only marginally more useful than UNKNOWN. And I think I prefer
> > UNKNOWN for now until we have a clear picture how these functions are
> > going to work. The main missing piece in this puzzle are properties that
> > allow us to register multiple functions and some decision as to what the
> > scope of 'functions' is going to be.
> > 
> > You mentioned that you have a few entities that are using this function,
> > but if you would specify the exact function of those entities, what would
> > the function name(s) be?
> 
> I'm not sure, otherwise I would have proposed more precise functions :-)
> 
> Two of the entities just apply look up tables. They can thus be used for
> various purpose, such as gamma correction, A-law/µ-law (de)compression, ...
> 
> The last entity is an interface between the VSP and the display device and
> just handles buffering, clock domain crossing and synchronization.
> 
> >> +	  <row>
> >> +
> >> <entry><constant>MEDIA_ENT_F_PROC_VIDEO_COMPOSER</constant></entry>
> >> +	    <entry>Video composer (blender). An entity capable of video
> >> +		   composing must have at least two sink pads and one source
> >> +		   pad, and composes input video frames onto output video
> >> +		   frames. Composition can be performed using alpha blending,
> >> +		   color keying, raster operations (ROP), stitching or any other
> >> +		   means.
> >> +	    </entry>
> >> +	  </row>
> > 
> > This one looks OK to me.
> > 
> >> +	  </row>
> >> +
> >> <entry><constant>MEDIA_ENT_F_PROC_VIDEO_CONVERTER</constant></entry>
> >> +	    <entry>Video format converter. An entity capable of video format
> >> +		   conversion must have at least one sink pad and one source
> >> +		   pad, and convert the format of pixels received on its sink
> >> +		   pad(s) to a different format output on its source pad(s).
> >> +	    </entry>
> >> +	  </row>
> > 
> > This is too vague as well, I think. You said that you don't consider
> > de-interlacing a converter function, but what about colorimetry
> > conversion? Debayer? 4:2:2 to 4:2:0 conversion or vice versa?
> 
> I'd consider that as video format conversion, yes.
> 
> The three entities that implement this function in the vsp1 driver are
> ARGB8888 <-> AHSV8888 converters and RGB <-> YUV converters (with various
> RGB and YUV formats supported).
> 
> >> +	  <row>
> >> +	    <entry><constant>MEDIA_ENT_F_PROC_VIDEO_SCALER</constant></entry>
> >> +	    <entry>Video scaler. An entity capable of video scaling must have
> >> +		   at least one sink pad and one source pad, and scaling the
> >> +		   video frame(s) received on its sink pad(s) to a different
> >> +		   resolution output on its source pad(s). The range of
> >> +		   supported scaling ratios is entity-specific and can differ
> >> +		   between the horizontal and vertical directions. In particular
> >> +		   scaling can be supported in one direction only.
> >> +	    </entry>
> >> +	  </row>
> > 
> > This looks OK too, although would sensor binning and/or skipping also be
> > considered scaling?
> 
> I would consider them as scaling, yes. Sakari, any opinion on that ?
> 
> >>  	</tbody>
> >>        </tgroup>
> >>      </table>

-- 
Regards,

Laurent Pinchart