[PATCH 0/3] Collapse vimc into single monolithic driver

[PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and
component structure added a level of complexity making it hard to
maintain without adding any real benefit.
    
The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to
collapse vimc into single monolithic driver first and then split the
sensor off as a separate module.

This patch series emoves the component API and makes minimal changes to
the code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off
as a separate module in the future.

Major design elements in this change are:
    - Use existing struct vimc_ent_config and struct vimc_pipeline_config
      to drive the initialization of the functional components.
    - Make vimc_ent_config global by moving it to vimc.h
    - Add two new hooks add and rm to initialize and register, unregister
      and free subdevs.
    - All component API is now gone and bind and unbind hooks are modified
      to do "add" and "rm" with minimal changes to just add and rm subdevs.
    - vimc-core's bind and unbind are now register and unregister.
    - vimc-core invokes "add" hooks from its vimc_register_devices().
      The "add" hooks remain the same and register subdevs. They don't
      create platform devices of their own and use vimc's pdev.dev as
      their reference device. The "add" hooks save their vimc_ent_device(s)
      in the corresponding vimc_ent_config.
    - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
      and cleanup.
    - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
      and the corresponding struct vimc_ent_config pointer.
    
The following configure and stream test works on all devices.
    
    media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
    media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
    
    v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
    v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
    v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
    
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
    v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

The third patch in the series fixes a general protection fault found
when rmmod is done while stream is active.

vimc_print_dot (--print-dot) topology after this change:
digraph board {
	rankdir=TB
	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000001:port0 -> n00000005:port0 [style=bold]
	n00000001:port0 -> n0000000b [style=bold]
	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000003:port0 -> n00000008:port0 [style=bold]
	n00000003:port0 -> n0000000f [style=bold]
	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000005:port1 -> n00000015:port0
	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000008:port1 -> n00000015:port0 [style=dashed]
	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000013:port0 -> n00000015:port0 [style=dashed]
	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
	n00000015:port1 -> n00000018 [style=bold]
	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
}

Shuah Khan (3):
  media: vimc: move private defines to a common header
  media: vimc: Collapse component structure into a single monolithic
    driver
  media: vimc: Fix gpf in rmmod path when stream is active

 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  96 ++----------
 drivers/media/platform/vimc/vimc-common.c  |   2 +-
 drivers/media/platform/vimc/vimc-core.c    | 174 +++++++--------------
 drivers/media/platform/vimc/vimc-debayer.c |  84 ++--------
 drivers/media/platform/vimc/vimc-scaler.c  |  83 ++--------
 drivers/media/platform/vimc/vimc-sensor.c  |  82 ++--------
 drivers/media/platform/vimc/vimc.h         | 121 ++++++++++++++
 8 files changed, 230 insertions(+), 419 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

-- 
2.17.1

[PATCH 1/3] media: vimc: move private defines to a common header

[Shuah Khan]

In preparation for collapsing the component driver structure into
a monolith, move private device structure defines to a new common
header file.

Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
---
 drivers/media/platform/vimc/vimc-capture.c |  21 +----
 drivers/media/platform/vimc/vimc-core.c    |  18 +---
 drivers/media/platform/vimc/vimc-debayer.c |  16 +---
 drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
 drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
 drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
 6 files changed, 107 insertions(+), 78 deletions(-)
 create mode 100644 drivers/media/platform/vimc/vimc.h

diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
index 664855708fdf..c52fc5d97c2d 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -13,6 +13,7 @@
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 #include "vimc-streamer.h"
 
@@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_SRGGB12,
 };
 
-struct vimc_cap_device {
-	struct vimc_ent_device ved;
-	struct video_device vdev;
-	struct device *dev;
-	struct v4l2_pix_format format;
-	struct vb2_queue queue;
-	struct list_head buf_list;
-	/*
-	 * NOTE: in a real driver, a spin lock must be used to access the
-	 * queue because the frames are generated from a hardware interruption
-	 * and the isr is not allowed to sleep.
-	 * Even if it is not necessary a spinlock in the vimc driver, we
-	 * use it here as a code reference
-	 */
-	spinlock_t qlock;
-	struct mutex lock;
-	u32 sequence;
-	struct vimc_stream stream;
-};
-
 static const struct v4l2_pix_format fmt_default = {
 	.width = 640,
 	.height = 480,
diff --git a/drivers/media/platform/vimc/vimc-core.c b/drivers/media/platform/vimc/vimc-core.c
index 571c55aa0e16..c9b351472272 100644
--- a/drivers/media/platform/vimc/vimc-core.c
+++ b/drivers/media/platform/vimc/vimc-core.c
@@ -12,6 +12,7 @@
 #include <media/media-device.h>
 #include <media/v4l2-device.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"
@@ -24,23 +25,6 @@
 	.flags = link_flags,					\
 }
 
-struct vimc_device {
-	/* The platform device */
-	struct platform_device pdev;
-
-	/* The pipeline configuration */
-	const struct vimc_pipeline_config *pipe_cfg;
-
-	/* The Associated media_device parent */
-	struct media_device mdev;
-
-	/* Internal v4l2 parent device*/
-	struct v4l2_device v4l2_dev;
-
-	/* Subdevices */
-	struct platform_device **subdevs;
-};
-
 /* Structure which describes individual configuration for each entity */
 struct vimc_ent_config {
 	const char *name;
diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
index 00598fbf3cba..750752bb173c 100644
--- a/drivers/media/platform/vimc/vimc-debayer.c
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_DEB_DRV_NAME "vimc-debayer"
@@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
 	enum vimc_deb_rgb_colors order[2][2];
 };
 
-struct vimc_deb_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* The active format */
-	struct v4l2_mbus_framefmt sink_fmt;
-	u32 src_code;
-	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
-			    unsigned int col, unsigned int rgb[3]);
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	const struct vimc_deb_pix_map *sink_pix_map;
-	unsigned int sink_bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,
diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
index c7123a45c55b..fe99b9102ada 100644
--- a/drivers/media/platform/vimc/vimc-scaler.c
+++ b/drivers/media/platform/vimc/vimc-scaler.c
@@ -13,6 +13,7 @@
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SCA_DRV_NAME "vimc-scaler"
@@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
 	V4L2_PIX_FMT_ARGB32,
 };
 
-struct vimc_sca_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	/* NOTE: the source fmt is the same as the sink
-	 * with the width and hight multiplied by mult
-	 */
-	struct v4l2_mbus_framefmt sink_fmt;
-	/* Values calculated when the stream starts */
-	u8 *src_frame;
-	unsigned int src_line_size;
-	unsigned int bpp;
-};
-
 static const struct v4l2_mbus_framefmt sink_fmt_default = {
 	.width = 640,
 	.height = 480,
diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
index 51359472eef2..6c57b1e262f9 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -16,22 +16,11 @@
 #include <media/v4l2-subdev.h>
 #include <media/tpg/v4l2-tpg.h>
 
+#include "vimc.h"
 #include "vimc-common.h"
 
 #define VIMC_SEN_DRV_NAME "vimc-sensor"
 
-struct vimc_sen_device {
-	struct vimc_ent_device ved;
-	struct v4l2_subdev sd;
-	struct device *dev;
-	struct tpg_data tpg;
-	struct task_struct *kthread_sen;
-	u8 *frame;
-	/* The active format */
-	struct v4l2_mbus_framefmt mbus_format;
-	struct v4l2_ctrl_handler hdl;
-};
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,
diff --git a/drivers/media/platform/vimc/vimc.h b/drivers/media/platform/vimc/vimc.h
new file mode 100644
index 000000000000..a5adebdda941
--- /dev/null
+++ b/drivers/media/platform/vimc/vimc.h
@@ -0,0 +1,102 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ *
+ * Copyright (C) 2019 Shuah Khan <skhan@linuxfoundation.org>
+ *
+ */
+
+/*
+ * This file defines vimc driver device and sub-device structures.
+ */
+
+#ifndef _VIMC_H_
+#define _VIMC_H_
+
+#include <linux/platform_device.h>
+#include <media/media-device.h>
+#include <media/v4l2-device.h>
+#include <media/videobuf2-core.h>
+#include <media/tpg/v4l2-tpg.h>
+#include <media/v4l2-ctrls.h>
+
+#include "vimc-common.h"
+
+struct vimc_cap_device {
+	struct vimc_ent_device ved;
+	struct video_device vdev;
+	struct device *dev;
+	struct v4l2_pix_format format;
+	struct vb2_queue queue;
+	struct list_head buf_list;
+	/*
+	 * NOTE: in a real driver, a spin lock must be used to access the
+	 * queue because the frames are generated from a hardware interruption
+	 * and the isr is not allowed to sleep.
+	 * Even if it is not necessary a spinlock in the vimc driver, we
+	 * use it here as a code reference
+	 */
+	spinlock_t qlock;
+	struct mutex lock;
+	u32 sequence;
+	struct vimc_stream stream;
+};
+
+struct vimc_sca_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* NOTE: the source fmt is the same as the sink
+	 * with the width and hight multiplied by mult
+	 */
+	struct v4l2_mbus_framefmt sink_fmt;
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	unsigned int src_line_size;
+	unsigned int bpp;
+};
+
+struct vimc_deb_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	/* The active format */
+	struct v4l2_mbus_framefmt sink_fmt;
+	u32 src_code;
+	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
+			    unsigned int col, unsigned int rgb[3]);
+	/* Values calculated when the stream starts */
+	u8 *src_frame;
+	const struct vimc_deb_pix_map *sink_pix_map;
+	unsigned int sink_bpp;
+};
+
+struct vimc_sen_device {
+	struct vimc_ent_device ved;
+	struct v4l2_subdev sd;
+	struct device *dev;
+	struct tpg_data tpg;
+	struct task_struct *kthread_sen;
+	u8 *frame;
+	/* The active format */
+	struct v4l2_mbus_framefmt mbus_format;
+	struct v4l2_ctrl_handler hdl;
+};
+
+struct vimc_device {
+	/* The platform device */
+	struct platform_device pdev;
+
+	/* The pipeline configuration */
+	const struct vimc_pipeline_config *pipe_cfg;
+
+	/* The Associated media_device parent */
+	struct media_device mdev;
+
+	/* Internal v4l2 parent device*/
+	struct v4l2_device v4l2_dev;
+
+	/* Subdevices */
+	struct platform_device **subdevs;
+};
+
+#endif
-- 
2.17.1

Re: [PATCH 1/3] media: vimc: move private defines to a common header

[Helen Koike]

Hi Shuah,

Thanks for the patch.

On 8/9/19 6:45 PM, Shuah Khan wrote:
> In preparation for collapsing the component driver structure into
> a monolith, move private device structure defines to a new common
> header file.
> 
> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
> ---
>  drivers/media/platform/vimc/vimc-capture.c |  21 +----
>  drivers/media/platform/vimc/vimc-core.c    |  18 +---
>  drivers/media/platform/vimc/vimc-debayer.c |  16 +---
>  drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
>  drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
>  drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
>  6 files changed, 107 insertions(+), 78 deletions(-)
>  create mode 100644 drivers/media/platform/vimc/vimc.h
> 
> diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
> index 664855708fdf..c52fc5d97c2d 100644
> --- a/drivers/media/platform/vimc/vimc-capture.c
> +++ b/drivers/media/platform/vimc/vimc-capture.c
> @@ -13,6 +13,7 @@
>  #include <media/videobuf2-core.h>
>  #include <media/videobuf2-vmalloc.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  #include "vimc-streamer.h"
>  
> @@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
>  	V4L2_PIX_FMT_SRGGB12,
>  };
>  
> -struct vimc_cap_device {
> -	struct vimc_ent_device ved;
> -	struct video_device vdev;
> -	struct device *dev;
> -	struct v4l2_pix_format format;
> -	struct vb2_queue queue;
> -	struct list_head buf_list;
> -	/*
> -	 * NOTE: in a real driver, a spin lock must be used to access the
> -	 * queue because the frames are generated from a hardware interruption
> -	 * and the isr is not allowed to sleep.
> -	 * Even if it is not necessary a spinlock in the vimc driver, we
> -	 * use it here as a code reference
> -	 */
> -	spinlock_t qlock;
> -	struct mutex lock;
> -	u32 sequence;
> -	struct vimc_stream stream;
> -};
> -
>  static const struct v4l2_pix_format fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc-core.c b/drivers/media/platform/vimc/vimc-core.c
> index 571c55aa0e16..c9b351472272 100644
> --- a/drivers/media/platform/vimc/vimc-core.c
> +++ b/drivers/media/platform/vimc/vimc-core.c
> @@ -12,6 +12,7 @@
>  #include <media/media-device.h>
>  #include <media/v4l2-device.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"
> @@ -24,23 +25,6 @@
>  	.flags = link_flags,					\
>  }
>  
> -struct vimc_device {
> -	/* The platform device */
> -	struct platform_device pdev;
> -
> -	/* The pipeline configuration */
> -	const struct vimc_pipeline_config *pipe_cfg;
> -
> -	/* The Associated media_device parent */
> -	struct media_device mdev;
> -
> -	/* Internal v4l2 parent device*/
> -	struct v4l2_device v4l2_dev;
> -
> -	/* Subdevices */
> -	struct platform_device **subdevs;
> -};
> -
>  /* Structure which describes individual configuration for each entity */
>  struct vimc_ent_config {
>  	const char *name;
> diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
> index 00598fbf3cba..750752bb173c 100644
> --- a/drivers/media/platform/vimc/vimc-debayer.c
> +++ b/drivers/media/platform/vimc/vimc-debayer.c
> @@ -13,6 +13,7 @@
>  #include <linux/v4l2-mediabus.h>
>  #include <media/v4l2-subdev.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_DEB_DRV_NAME "vimc-debayer"
> @@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
>  	enum vimc_deb_rgb_colors order[2][2];
>  };
>  
> -struct vimc_deb_device {
> -	struct vimc_ent_device ved;
> -	struct v4l2_subdev sd;
> -	struct device *dev;
> -	/* The active format */
> -	struct v4l2_mbus_framefmt sink_fmt;
> -	u32 src_code;
> -	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
> -			    unsigned int col, unsigned int rgb[3]);
> -	/* Values calculated when the stream starts */
> -	u8 *src_frame;
> -	const struct vimc_deb_pix_map *sink_pix_map;
> -	unsigned int sink_bpp;
> -};
> -
>  static const struct v4l2_mbus_framefmt sink_fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
> index c7123a45c55b..fe99b9102ada 100644
> --- a/drivers/media/platform/vimc/vimc-scaler.c
> +++ b/drivers/media/platform/vimc/vimc-scaler.c
> @@ -13,6 +13,7 @@
>  #include <linux/v4l2-mediabus.h>
>  #include <media/v4l2-subdev.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_SCA_DRV_NAME "vimc-scaler"
> @@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
>  	V4L2_PIX_FMT_ARGB32,
>  };
>  
> -struct vimc_sca_device {
> -	struct vimc_ent_device ved;
> -	struct v4l2_subdev sd;
> -	struct device *dev;
> -	/* NOTE: the source fmt is the same as the sink
> -	 * with the width and hight multiplied by mult
> -	 */
> -	struct v4l2_mbus_framefmt sink_fmt;
> -	/* Values calculated when the stream starts */
> -	u8 *src_frame;
> -	unsigned int src_line_size;
> -	unsigned int bpp;
> -};
> -
>  static const struct v4l2_mbus_framefmt sink_fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
> index 51359472eef2..6c57b1e262f9 100644
> --- a/drivers/media/platform/vimc/vimc-sensor.c
> +++ b/drivers/media/platform/vimc/vimc-sensor.c
> @@ -16,22 +16,11 @@
>  #include <media/v4l2-subdev.h>
>  #include <media/tpg/v4l2-tpg.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_SEN_DRV_NAME "vimc-sensor"
>  
> -struct vimc_sen_device {
> -	struct vimc_ent_device ved;
> -	struct v4l2_subdev sd;
> -	struct device *dev;
> -	struct tpg_data tpg;
> -	struct task_struct *kthread_sen;
> -	u8 *frame;
> -	/* The active format */
> -	struct v4l2_mbus_framefmt mbus_format;
> -	struct v4l2_ctrl_handler hdl;
> -};
> -
>  static const struct v4l2_mbus_framefmt fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc.h b/drivers/media/platform/vimc/vimc.h
> new file mode 100644
> index 000000000000..a5adebdda941
> --- /dev/null
> +++ b/drivers/media/platform/vimc/vimc.h

I was wondering if instead of creating a vimc.h, if we could move
to vimc-common.h,just because it is not clear to me the difference
between the two now, what do you think?

Regards,
Helen

> @@ -0,0 +1,102 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + *
> + * Copyright (C) 2019 Shuah Khan <skhan@linuxfoundation.org>
> + *
> + */
> +
> +/*
> + * This file defines vimc driver device and sub-device structures.
> + */
> +
> +#ifndef _VIMC_H_
> +#define _VIMC_H_
> +
> +#include <linux/platform_device.h>
> +#include <media/media-device.h>
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-core.h>
> +#include <media/tpg/v4l2-tpg.h>
> +#include <media/v4l2-ctrls.h>
> +
> +#include "vimc-common.h"
> +
> +struct vimc_cap_device {
> +	struct vimc_ent_device ved;
> +	struct video_device vdev;
> +	struct device *dev;
> +	struct v4l2_pix_format format;
> +	struct vb2_queue queue;
> +	struct list_head buf_list;
> +	/*
> +	 * NOTE: in a real driver, a spin lock must be used to access the
> +	 * queue because the frames are generated from a hardware interruption
> +	 * and the isr is not allowed to sleep.
> +	 * Even if it is not necessary a spinlock in the vimc driver, we
> +	 * use it here as a code reference
> +	 */
> +	spinlock_t qlock;
> +	struct mutex lock;
> +	u32 sequence;
> +	struct vimc_stream stream;
> +};
> +
> +struct vimc_sca_device {
> +	struct vimc_ent_device ved;
> +	struct v4l2_subdev sd;
> +	struct device *dev;
> +	/* NOTE: the source fmt is the same as the sink
> +	 * with the width and hight multiplied by mult
> +	 */
> +	struct v4l2_mbus_framefmt sink_fmt;
> +	/* Values calculated when the stream starts */
> +	u8 *src_frame;
> +	unsigned int src_line_size;
> +	unsigned int bpp;
> +};
> +
> +struct vimc_deb_device {
> +	struct vimc_ent_device ved;
> +	struct v4l2_subdev sd;
> +	struct device *dev;
> +	/* The active format */
> +	struct v4l2_mbus_framefmt sink_fmt;
> +	u32 src_code;
> +	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
> +			    unsigned int col, unsigned int rgb[3]);
> +	/* Values calculated when the stream starts */
> +	u8 *src_frame;
> +	const struct vimc_deb_pix_map *sink_pix_map;
> +	unsigned int sink_bpp;
> +};
> +
> +struct vimc_sen_device {
> +	struct vimc_ent_device ved;
> +	struct v4l2_subdev sd;
> +	struct device *dev;
> +	struct tpg_data tpg;
> +	struct task_struct *kthread_sen;
> +	u8 *frame;
> +	/* The active format */
> +	struct v4l2_mbus_framefmt mbus_format;
> +	struct v4l2_ctrl_handler hdl;
> +};
> +
> +struct vimc_device {
> +	/* The platform device */
> +	struct platform_device pdev;
> +
> +	/* The pipeline configuration */
> +	const struct vimc_pipeline_config *pipe_cfg;
> +
> +	/* The Associated media_device parent */
> +	struct media_device mdev;
> +
> +	/* Internal v4l2 parent device*/
> +	struct v4l2_device v4l2_dev;
> +
> +	/* Subdevices */
> +	struct platform_device **subdevs;
> +};
> +
> +#endif
> 

Re: [PATCH 1/3] media: vimc: move private defines to a common header

[Shuah Khan]

Hi Helen,

On 8/9/19 9:15 PM, Helen Koike wrote:
> Hi Shuah,
> 
> Thanks for the patch.
> 
> On 8/9/19 6:45 PM, Shuah Khan wrote:
>> In preparation for collapsing the component driver structure into
>> a monolith, move private device structure defines to a new common
>> header file.
>>
>> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
>> ---
>>   drivers/media/platform/vimc/vimc-capture.c |  21 +----
>>   drivers/media/platform/vimc/vimc-core.c    |  18 +---
>>   drivers/media/platform/vimc/vimc-debayer.c |  16 +---
>>   drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
>>   drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
>>   drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
>>   6 files changed, 107 insertions(+), 78 deletions(-)
>>   create mode 100644 drivers/media/platform/vimc/vimc.h
>>
>> diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
>> index 664855708fdf..c52fc5d97c2d 100644
>> --- a/drivers/media/platform/vimc/vimc-capture.c
>> +++ b/drivers/media/platform/vimc/vimc-capture.c
>> @@ -13,6 +13,7 @@
>>   #include <media/videobuf2-core.h>
>>   #include <media/videobuf2-vmalloc.h>
>>   
>> +#include "vimc.h"
>>   #include "vimc-common.h"
>>   #include "vimc-streamer.h"
>>   
>> @@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
>>   	V4L2_PIX_FMT_SRGGB12,
>>   };
>>   
>> -struct vimc_cap_device {
>> -	struct vimc_ent_device ved;
>> -	struct video_device vdev;
>> -	struct device *dev;
>> -	struct v4l2_pix_format format;
>> -	struct vb2_queue queue;
>> -	struct list_head buf_list;
>> -	/*
>> -	 * NOTE: in a real driver, a spin lock must be used to access the
>> -	 * queue because the frames are generated from a hardware interruption
>> -	 * and the isr is not allowed to sleep.
>> -	 * Even if it is not necessary a spinlock in the vimc driver, we
>> -	 * use it here as a code reference
>> -	 */
>> -	spinlock_t qlock;
>> -	struct mutex lock;
>> -	u32 sequence;
>> -	struct vimc_stream stream;
>> -};
>> -
>>   static const struct v4l2_pix_format fmt_default = {
>>   	.width = 640,
>>   	.height = 480,
>> diff --git a/drivers/media/platform/vimc/vimc-core.c b/drivers/media/platform/vimc/vimc-core.c
>> index 571c55aa0e16..c9b351472272 100644
>> --- a/drivers/media/platform/vimc/vimc-core.c
>> +++ b/drivers/media/platform/vimc/vimc-core.c
>> @@ -12,6 +12,7 @@
>>   #include <media/media-device.h>
>>   #include <media/v4l2-device.h>
>>   
>> +#include "vimc.h"
>>   #include "vimc-common.h"
>>   
>>   #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"
>> @@ -24,23 +25,6 @@
>>   	.flags = link_flags,					\
>>   }
>>   
>> -struct vimc_device {
>> -	/* The platform device */
>> -	struct platform_device pdev;
>> -
>> -	/* The pipeline configuration */
>> -	const struct vimc_pipeline_config *pipe_cfg;
>> -
>> -	/* The Associated media_device parent */
>> -	struct media_device mdev;
>> -
>> -	/* Internal v4l2 parent device*/
>> -	struct v4l2_device v4l2_dev;
>> -
>> -	/* Subdevices */
>> -	struct platform_device **subdevs;
>> -};
>> -
>>   /* Structure which describes individual configuration for each entity */
>>   struct vimc_ent_config {
>>   	const char *name;
>> diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
>> index 00598fbf3cba..750752bb173c 100644
>> --- a/drivers/media/platform/vimc/vimc-debayer.c
>> +++ b/drivers/media/platform/vimc/vimc-debayer.c
>> @@ -13,6 +13,7 @@
>>   #include <linux/v4l2-mediabus.h>
>>   #include <media/v4l2-subdev.h>
>>   
>> +#include "vimc.h"
>>   #include "vimc-common.h"
>>   
>>   #define VIMC_DEB_DRV_NAME "vimc-debayer"
>> @@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
>>   	enum vimc_deb_rgb_colors order[2][2];
>>   };
>>   
>> -struct vimc_deb_device {
>> -	struct vimc_ent_device ved;
>> -	struct v4l2_subdev sd;
>> -	struct device *dev;
>> -	/* The active format */
>> -	struct v4l2_mbus_framefmt sink_fmt;
>> -	u32 src_code;
>> -	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
>> -			    unsigned int col, unsigned int rgb[3]);
>> -	/* Values calculated when the stream starts */
>> -	u8 *src_frame;
>> -	const struct vimc_deb_pix_map *sink_pix_map;
>> -	unsigned int sink_bpp;
>> -};
>> -
>>   static const struct v4l2_mbus_framefmt sink_fmt_default = {
>>   	.width = 640,
>>   	.height = 480,
>> diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
>> index c7123a45c55b..fe99b9102ada 100644
>> --- a/drivers/media/platform/vimc/vimc-scaler.c
>> +++ b/drivers/media/platform/vimc/vimc-scaler.c
>> @@ -13,6 +13,7 @@
>>   #include <linux/v4l2-mediabus.h>
>>   #include <media/v4l2-subdev.h>
>>   
>> +#include "vimc.h"
>>   #include "vimc-common.h"
>>   
>>   #define VIMC_SCA_DRV_NAME "vimc-scaler"
>> @@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
>>   	V4L2_PIX_FMT_ARGB32,
>>   };
>>   
>> -struct vimc_sca_device {
>> -	struct vimc_ent_device ved;
>> -	struct v4l2_subdev sd;
>> -	struct device *dev;
>> -	/* NOTE: the source fmt is the same as the sink
>> -	 * with the width and hight multiplied by mult
>> -	 */
>> -	struct v4l2_mbus_framefmt sink_fmt;
>> -	/* Values calculated when the stream starts */
>> -	u8 *src_frame;
>> -	unsigned int src_line_size;
>> -	unsigned int bpp;
>> -};
>> -
>>   static const struct v4l2_mbus_framefmt sink_fmt_default = {
>>   	.width = 640,
>>   	.height = 480,
>> diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
>> index 51359472eef2..6c57b1e262f9 100644
>> --- a/drivers/media/platform/vimc/vimc-sensor.c
>> +++ b/drivers/media/platform/vimc/vimc-sensor.c
>> @@ -16,22 +16,11 @@
>>   #include <media/v4l2-subdev.h>
>>   #include <media/tpg/v4l2-tpg.h>
>>   
>> +#include "vimc.h"
>>   #include "vimc-common.h"
>>   
>>   #define VIMC_SEN_DRV_NAME "vimc-sensor"
>>   
>> -struct vimc_sen_device {
>> -	struct vimc_ent_device ved;
>> -	struct v4l2_subdev sd;
>> -	struct device *dev;
>> -	struct tpg_data tpg;
>> -	struct task_struct *kthread_sen;
>> -	u8 *frame;
>> -	/* The active format */
>> -	struct v4l2_mbus_framefmt mbus_format;
>> -	struct v4l2_ctrl_handler hdl;
>> -};
>> -
>>   static const struct v4l2_mbus_framefmt fmt_default = {
>>   	.width = 640,
>>   	.height = 480,
>> diff --git a/drivers/media/platform/vimc/vimc.h b/drivers/media/platform/vimc/vimc.h
>> new file mode 100644
>> index 000000000000..a5adebdda941
>> --- /dev/null
>> +++ b/drivers/media/platform/vimc/vimc.h
> 
> I was wondering if instead of creating a vimc.h, if we could move
> to vimc-common.h,just because it is not clear to me the difference
> between the two now, what do you think?
> 

My thinking is that vimc-common.h is common for all the subdevs and
putting vimc-core defines and structures it shares it with the subdev
files can be in a separate file.

It is more of design choice to keep structures and defined organized.
Originally I was thinking all the subdev device structires need to be
global, and my patch set I sent out as such doesn't need that. I just
overlooked that when I sent the patches out.

This reduces the number of things that need to be common, I don't really
have any strong reasons for either choice of adding common defines to
vimc-common.h vs vimc.h - maybe with a slight tilt towards liking vimc.h

thanks,
-- Shuah

Re: [PATCH 1/3] media: vimc: move private defines to a common header

[Laurent Pinchart]

Hi Shuah,

Thank you for the patch.

On Fri, Aug 09, 2019 at 03:45:41PM -0600, Shuah Khan wrote:
> In preparation for collapsing the component driver structure into
> a monolith, move private device structure defines to a new common
> header file.

Apart from the vimc_device structure, this doesn't seem to be needed.
I'd rather keep each structure private to the .c file that handles it,
and only share vimc_device globally.

> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
> ---
>  drivers/media/platform/vimc/vimc-capture.c |  21 +----
>  drivers/media/platform/vimc/vimc-core.c    |  18 +---
>  drivers/media/platform/vimc/vimc-debayer.c |  16 +---
>  drivers/media/platform/vimc/vimc-scaler.c  |  15 +--
>  drivers/media/platform/vimc/vimc-sensor.c  |  13 +--
>  drivers/media/platform/vimc/vimc.h         | 102 +++++++++++++++++++++
>  6 files changed, 107 insertions(+), 78 deletions(-)
>  create mode 100644 drivers/media/platform/vimc/vimc.h
> 
> diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
> index 664855708fdf..c52fc5d97c2d 100644
> --- a/drivers/media/platform/vimc/vimc-capture.c
> +++ b/drivers/media/platform/vimc/vimc-capture.c
> @@ -13,6 +13,7 @@
>  #include <media/videobuf2-core.h>
>  #include <media/videobuf2-vmalloc.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  #include "vimc-streamer.h"
>  
> @@ -44,26 +45,6 @@ static const u32 vimc_cap_supported_pixfmt[] = {
>  	V4L2_PIX_FMT_SRGGB12,
>  };
>  
> -struct vimc_cap_device {
> -	struct vimc_ent_device ved;
> -	struct video_device vdev;
> -	struct device *dev;
> -	struct v4l2_pix_format format;
> -	struct vb2_queue queue;
> -	struct list_head buf_list;
> -	/*
> -	 * NOTE: in a real driver, a spin lock must be used to access the
> -	 * queue because the frames are generated from a hardware interruption
> -	 * and the isr is not allowed to sleep.
> -	 * Even if it is not necessary a spinlock in the vimc driver, we
> -	 * use it here as a code reference
> -	 */
> -	spinlock_t qlock;
> -	struct mutex lock;
> -	u32 sequence;
> -	struct vimc_stream stream;
> -};
> -
>  static const struct v4l2_pix_format fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc-core.c b/drivers/media/platform/vimc/vimc-core.c
> index 571c55aa0e16..c9b351472272 100644
> --- a/drivers/media/platform/vimc/vimc-core.c
> +++ b/drivers/media/platform/vimc/vimc-core.c
> @@ -12,6 +12,7 @@
>  #include <media/media-device.h>
>  #include <media/v4l2-device.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_MDEV_MODEL_NAME "VIMC MDEV"
> @@ -24,23 +25,6 @@
>  	.flags = link_flags,					\
>  }
>  
> -struct vimc_device {
> -	/* The platform device */
> -	struct platform_device pdev;
> -
> -	/* The pipeline configuration */
> -	const struct vimc_pipeline_config *pipe_cfg;
> -
> -	/* The Associated media_device parent */
> -	struct media_device mdev;
> -
> -	/* Internal v4l2 parent device*/
> -	struct v4l2_device v4l2_dev;
> -
> -	/* Subdevices */
> -	struct platform_device **subdevs;
> -};
> -
>  /* Structure which describes individual configuration for each entity */
>  struct vimc_ent_config {
>  	const char *name;
> diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
> index 00598fbf3cba..750752bb173c 100644
> --- a/drivers/media/platform/vimc/vimc-debayer.c
> +++ b/drivers/media/platform/vimc/vimc-debayer.c
> @@ -13,6 +13,7 @@
>  #include <linux/v4l2-mediabus.h>
>  #include <media/v4l2-subdev.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_DEB_DRV_NAME "vimc-debayer"
> @@ -44,21 +45,6 @@ struct vimc_deb_pix_map {
>  	enum vimc_deb_rgb_colors order[2][2];
>  };
>  
> -struct vimc_deb_device {
> -	struct vimc_ent_device ved;
> -	struct v4l2_subdev sd;
> -	struct device *dev;
> -	/* The active format */
> -	struct v4l2_mbus_framefmt sink_fmt;
> -	u32 src_code;
> -	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
> -			    unsigned int col, unsigned int rgb[3]);
> -	/* Values calculated when the stream starts */
> -	u8 *src_frame;
> -	const struct vimc_deb_pix_map *sink_pix_map;
> -	unsigned int sink_bpp;
> -};
> -
>  static const struct v4l2_mbus_framefmt sink_fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
> index c7123a45c55b..fe99b9102ada 100644
> --- a/drivers/media/platform/vimc/vimc-scaler.c
> +++ b/drivers/media/platform/vimc/vimc-scaler.c
> @@ -13,6 +13,7 @@
>  #include <linux/v4l2-mediabus.h>
>  #include <media/v4l2-subdev.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_SCA_DRV_NAME "vimc-scaler"
> @@ -31,20 +32,6 @@ static const u32 vimc_sca_supported_pixfmt[] = {
>  	V4L2_PIX_FMT_ARGB32,
>  };
>  
> -struct vimc_sca_device {
> -	struct vimc_ent_device ved;
> -	struct v4l2_subdev sd;
> -	struct device *dev;
> -	/* NOTE: the source fmt is the same as the sink
> -	 * with the width and hight multiplied by mult
> -	 */
> -	struct v4l2_mbus_framefmt sink_fmt;
> -	/* Values calculated when the stream starts */
> -	u8 *src_frame;
> -	unsigned int src_line_size;
> -	unsigned int bpp;
> -};
> -
>  static const struct v4l2_mbus_framefmt sink_fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
> index 51359472eef2..6c57b1e262f9 100644
> --- a/drivers/media/platform/vimc/vimc-sensor.c
> +++ b/drivers/media/platform/vimc/vimc-sensor.c
> @@ -16,22 +16,11 @@
>  #include <media/v4l2-subdev.h>
>  #include <media/tpg/v4l2-tpg.h>
>  
> +#include "vimc.h"
>  #include "vimc-common.h"
>  
>  #define VIMC_SEN_DRV_NAME "vimc-sensor"
>  
> -struct vimc_sen_device {
> -	struct vimc_ent_device ved;
> -	struct v4l2_subdev sd;
> -	struct device *dev;
> -	struct tpg_data tpg;
> -	struct task_struct *kthread_sen;
> -	u8 *frame;
> -	/* The active format */
> -	struct v4l2_mbus_framefmt mbus_format;
> -	struct v4l2_ctrl_handler hdl;
> -};
> -
>  static const struct v4l2_mbus_framefmt fmt_default = {
>  	.width = 640,
>  	.height = 480,
> diff --git a/drivers/media/platform/vimc/vimc.h b/drivers/media/platform/vimc/vimc.h
> new file mode 100644
> index 000000000000..a5adebdda941
> --- /dev/null
> +++ b/drivers/media/platform/vimc/vimc.h
> @@ -0,0 +1,102 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + *
> + * Copyright (C) 2019 Shuah Khan <skhan@linuxfoundation.org>
> + *
> + */
> +
> +/*
> + * This file defines vimc driver device and sub-device structures.
> + */
> +
> +#ifndef _VIMC_H_
> +#define _VIMC_H_
> +
> +#include <linux/platform_device.h>
> +#include <media/media-device.h>
> +#include <media/v4l2-device.h>
> +#include <media/videobuf2-core.h>
> +#include <media/tpg/v4l2-tpg.h>
> +#include <media/v4l2-ctrls.h>
> +
> +#include "vimc-common.h"
> +
> +struct vimc_cap_device {
> +	struct vimc_ent_device ved;
> +	struct video_device vdev;
> +	struct device *dev;
> +	struct v4l2_pix_format format;
> +	struct vb2_queue queue;
> +	struct list_head buf_list;
> +	/*
> +	 * NOTE: in a real driver, a spin lock must be used to access the
> +	 * queue because the frames are generated from a hardware interruption
> +	 * and the isr is not allowed to sleep.
> +	 * Even if it is not necessary a spinlock in the vimc driver, we
> +	 * use it here as a code reference
> +	 */
> +	spinlock_t qlock;
> +	struct mutex lock;
> +	u32 sequence;
> +	struct vimc_stream stream;
> +};
> +
> +struct vimc_sca_device {
> +	struct vimc_ent_device ved;
> +	struct v4l2_subdev sd;
> +	struct device *dev;
> +	/* NOTE: the source fmt is the same as the sink
> +	 * with the width and hight multiplied by mult
> +	 */
> +	struct v4l2_mbus_framefmt sink_fmt;
> +	/* Values calculated when the stream starts */
> +	u8 *src_frame;
> +	unsigned int src_line_size;
> +	unsigned int bpp;
> +};
> +
> +struct vimc_deb_device {
> +	struct vimc_ent_device ved;
> +	struct v4l2_subdev sd;
> +	struct device *dev;
> +	/* The active format */
> +	struct v4l2_mbus_framefmt sink_fmt;
> +	u32 src_code;
> +	void (*set_rgb_src)(struct vimc_deb_device *vdeb, unsigned int lin,
> +			    unsigned int col, unsigned int rgb[3]);
> +	/* Values calculated when the stream starts */
> +	u8 *src_frame;
> +	const struct vimc_deb_pix_map *sink_pix_map;
> +	unsigned int sink_bpp;
> +};
> +
> +struct vimc_sen_device {
> +	struct vimc_ent_device ved;
> +	struct v4l2_subdev sd;
> +	struct device *dev;
> +	struct tpg_data tpg;
> +	struct task_struct *kthread_sen;
> +	u8 *frame;
> +	/* The active format */
> +	struct v4l2_mbus_framefmt mbus_format;
> +	struct v4l2_ctrl_handler hdl;
> +};
> +
> +struct vimc_device {
> +	/* The platform device */
> +	struct platform_device pdev;
> +
> +	/* The pipeline configuration */
> +	const struct vimc_pipeline_config *pipe_cfg;
> +
> +	/* The Associated media_device parent */
> +	struct media_device mdev;
> +
> +	/* Internal v4l2 parent device*/
> +	struct v4l2_device v4l2_dev;
> +
> +	/* Subdevices */
> +	struct platform_device **subdevs;
> +};
> +
> +#endif

-- 
Regards,

Laurent Pinchart

Re: [PATCH 1/3] media: vimc: move private defines to a common header

[Shuah Khan]

Hi Laurent,

On 8/10/19 8:14 AM, Laurent Pinchart wrote:
> Hi Shuah,
> 
> Thank you for the patch.
> 
> On Fri, Aug 09, 2019 at 03:45:41PM -0600, Shuah Khan wrote:
>> In preparation for collapsing the component driver structure into
>> a monolith, move private device structure defines to a new common
>> header file.
> 
> Apart from the vimc_device structure, this doesn't seem to be needed.
> I'd rather keep each structure private to the .c file that handles it,
> and only share vimc_device globally.
> 

Right. I initially thought that I needed these global. Once I completed
the patches without needing these as global, I overlooked updating the
patches.

I will take care of that. Any thoughts on vimc.h vs. adding vimc_device
struct to existing vimc-common.h

As I explained to Helen in response to her comment about:

"My thinking is that vimc-common.h is common for all the subdevs and
putting vimc-core defines and structures it shares it with the subdev
files can be in a separate file.

It is more of design choice to keep structures and defined organized.
Originally I was thinking all the subdev device structires need to be
global, and my patch set I sent out as such doesn't need that. I just
overlooked that when I sent the patches out.

This reduces the number of things that need to be common, I don't really
have any strong reasons for either choice of adding common defines to
vimc-common.h vs vimc.h - maybe with a slight tilt towards vimc.h"

Thanks all for a quick review and testing. I will work on v2 with your
comments. I want to make sure topology either looks the same as what
is in media master. I think it is, but I want to double check.

thanks,
-- Shuah

Re: [PATCH 1/3] media: vimc: move private defines to a common header

[Laurent Pinchart]

Hi Shua,

On Mon, Aug 12, 2019 at 08:19:27AM -0600, Shuah Khan wrote:
> On 8/10/19 8:14 AM, Laurent Pinchart wrote:
> > On Fri, Aug 09, 2019 at 03:45:41PM -0600, Shuah Khan wrote:
> >> In preparation for collapsing the component driver structure into
> >> a monolith, move private device structure defines to a new common
> >> header file.
> > 
> > Apart from the vimc_device structure, this doesn't seem to be needed.
> > I'd rather keep each structure private to the .c file that handles it,
> > and only share vimc_device globally.
> 
> Right. I initially thought that I needed these global. Once I completed
> the patches without needing these as global, I overlooked updating the
> patches.
> 
> I will take care of that. Any thoughts on vimc.h vs. adding vimc_device
> struct to existing vimc-common.h
> 
> As I explained to Helen in response to her comment about:
> 
> "My thinking is that vimc-common.h is common for all the subdevs and
> putting vimc-core defines and structures it shares it with the subdev
> files can be in a separate file.
> 
> It is more of design choice to keep structures and defined organized.
> Originally I was thinking all the subdev device structires need to be
> global, and my patch set I sent out as such doesn't need that. I just
> overlooked that when I sent the patches out.
> 
> This reduces the number of things that need to be common, I don't really
> have any strong reasons for either choice of adding common defines to
> vimc-common.h vs vimc.h - maybe with a slight tilt towards vimc.h"

The vimc_device structure fits nicely in vimc-common.h in my opinion, as
it's used by every component. I don't care much either way.

> Thanks all for a quick review and testing. I will work on v2 with your
> comments. I want to make sure topology either looks the same as what
> is in media master. I think it is, but I want to double check.

-- 
Regards,

Laurent Pinchart

Re: [PATCH 1/3] media: vimc: move private defines to a common header

[Shuah Khan]

On 8/12/19 8:24 AM, Laurent Pinchart wrote:
> Hi Shua,
> 
> On Mon, Aug 12, 2019 at 08:19:27AM -0600, Shuah Khan wrote:
>> On 8/10/19 8:14 AM, Laurent Pinchart wrote:
>>> On Fri, Aug 09, 2019 at 03:45:41PM -0600, Shuah Khan wrote:
>>>> In preparation for collapsing the component driver structure into
>>>> a monolith, move private device structure defines to a new common
>>>> header file.
>>>
>>> Apart from the vimc_device structure, this doesn't seem to be needed.
>>> I'd rather keep each structure private to the .c file that handles it,
>>> and only share vimc_device globally.
>>
>> Right. I initially thought that I needed these global. Once I completed
>> the patches without needing these as global, I overlooked updating the
>> patches.
>>
>> I will take care of that. Any thoughts on vimc.h vs. adding vimc_device
>> struct to existing vimc-common.h
>>
>> As I explained to Helen in response to her comment about:
>>
>> "My thinking is that vimc-common.h is common for all the subdevs and
>> putting vimc-core defines and structures it shares it with the subdev
>> files can be in a separate file.
>>
>> It is more of design choice to keep structures and defined organized.
>> Originally I was thinking all the subdev device structires need to be
>> global, and my patch set I sent out as such doesn't need that. I just
>> overlooked that when I sent the patches out.
>>
>> This reduces the number of things that need to be common, I don't really
>> have any strong reasons for either choice of adding common defines to
>> vimc-common.h vs vimc.h - maybe with a slight tilt towards vimc.h"
> 
> The vimc_device structure fits nicely in vimc-common.h in my opinion, as
> it's used by every component. I don't care much either way.
> 

Sounds good to me.

thanks,
-- Shuah

[PATCH 2/3] media: vimc: Collapse component structure into a single monolithic driver

[Shuah Khan]

vimc uses Component API to split the driver into functional components.
The real hardware resembles a monolith structure than component and
component structure added a level of complexity making it hard to
maintain without adding any real benefit.

The sensor is one vimc component that would makes sense to be a separate
module to closely align with the real hardware. It would be easier to
collapse vimc into single monolithic driver first and then split the
sensor off as a separate module.

Collapse it into a single monolithic driver removing the Component API.
This patch removes the component API and makes minimal changes to the
code base preserving the functional division of the code structure.
Preserving the functional structure allows us to split the sensor off
as a separate module in the future.

Major design elements in this change are:
- Use existing struct vimc_ent_config and struct vimc_pipeline_config
  to drive the initialization of the functional components.
- Make vimc_ent_config global by moving it to vimc.h
- Add two new hooks add and rm to initialize and register, unregister
  and free subdevs.
- All component API is now gone and bind and unbind hooks are modified
  to do "add" and "rm" with minimal changes to just add and rm subdevs.
- vimc-core's bind and unbind are now register and unregister.
- vimc-core invokes "add" hooks from its vimc_register_devices().
  The "add" hooks remain the same and register subdevs. They don't
  create platform devices of their own and use vimc's pdev.dev as
  their reference device. The "add" hooks save their vimc_ent_device(s)
  in the corresponding vimc_ent_config.
- vimc-core invokes "rm" hooks from its unregister to unregister subdevs
  and cleanup.
- vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
  and the corresponding struct vimc_ent_config pointer.

The following configure and stream test works on all devices.

media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'

v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81

v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3

Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
---
 drivers/media/platform/vimc/Makefile       |   7 +-
 drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
 drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
 drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
 drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
 drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
 drivers/media/platform/vimc/vimc.h         |  25 +++-
 7 files changed, 125 insertions(+), 343 deletions(-)

diff --git a/drivers/media/platform/vimc/Makefile b/drivers/media/platform/vimc/Makefile
index 96d06f030c31..a53b2b532e9f 100644
--- a/drivers/media/platform/vimc/Makefile
+++ b/drivers/media/platform/vimc/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0
-vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
+vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
+		vimc-debayer.o vimc-scaler.o vimc-sensor.o
+
+obj-$(CONFIG_VIDEO_VIMC) += vimc.o
 
-obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
-                vimc-scaler.o vimc-sensor.o
diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
index c52fc5d97c2d..b7b2d3c3d4f8 100644
--- a/drivers/media/platform/vimc/vimc-capture.c
+++ b/drivers/media/platform/vimc/vimc-capture.c
@@ -5,10 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
  */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <media/v4l2-ioctl.h>
 #include <media/videobuf2-core.h>
 #include <media/videobuf2-vmalloc.h>
@@ -17,8 +13,6 @@
 #include "vimc-common.h"
 #include "vimc-streamer.h"
 
-#define VIMC_CAP_DRV_NAME "vimc-capture"
-
 static const u32 vimc_cap_supported_pixfmt[] = {
 	V4L2_PIX_FMT_BGR24,
 	V4L2_PIX_FMT_RGB24,
@@ -348,11 +342,11 @@ static void vimc_cap_release(struct video_device *vdev)
 	kfree(vcap);
 }
 
-static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_cap_device *vcap = container_of(ved,
+						    struct vimc_cap_device,
 						    ved);
 
 	vb2_queue_release(&vcap->queue);
@@ -399,11 +393,9 @@ static void *vimc_cap_process_frame(struct vimc_ent_device *ved,
 	return NULL;
 }
 
-static int vimc_cap_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_cap_device *vcap;
 	struct video_device *vdev;
 	struct vb2_queue *q;
@@ -423,7 +415,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	}
 
 	/* Initialize the media entity */
-	vcap->vdev.entity.name = pdata->entity_name;
+	vcap->vdev.entity.name = vent->name;
 	vcap->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
 	ret = media_entity_pads_init(&vcap->vdev.entity,
 				     1, vcap->ved.pads);
@@ -447,8 +439,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
 	ret = vb2_queue_init(q);
 	if (ret) {
-		dev_err(comp, "%s: vb2 queue init failed (err=%d)\n",
-			pdata->entity_name, ret);
+		dev_err(&vimc->pdev.dev, "%s: vb2 queue init failed (err=%d)\n",
+			vent->name, ret);
 		goto err_clean_m_ent;
 	}
 
@@ -465,8 +457,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	vcap->ved.ent = &vcap->vdev.entity;
 	vcap->ved.process_frame = vimc_cap_process_frame;
 	vcap->ved.vdev_get_format = vimc_cap_get_format;
-	dev_set_drvdata(comp, &vcap->ved);
-	vcap->dev = comp;
+	vcap->dev = &vimc->pdev.dev;
 
 	/* Initialize the video_device struct */
 	vdev = &vcap->vdev;
@@ -479,17 +470,18 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 	vdev->queue = q;
 	vdev->v4l2_dev = v4l2_dev;
 	vdev->vfl_dir = VFL_DIR_RX;
-	strscpy(vdev->name, pdata->entity_name, sizeof(vdev->name));
+	strscpy(vdev->name, vent->name, sizeof(vdev->name));
 	video_set_drvdata(vdev, &vcap->ved);
 
 	/* Register the video_device with the v4l2 and the media framework */
 	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
 	if (ret) {
-		dev_err(comp, "%s: video register failed (err=%d)\n",
+		dev_err(&vimc->pdev.dev, "%s: video register failed (err=%d)\n",
 			vcap->vdev.name, ret);
 		goto err_release_queue;
 	}
 
+	vent->ved = &vcap->ved;
 	return 0;
 
 err_release_queue:
@@ -503,44 +495,3 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
 
 	return ret;
 }
-
-static const struct component_ops vimc_cap_comp_ops = {
-	.bind = vimc_cap_comp_bind,
-	.unbind = vimc_cap_comp_unbind,
-};
-
-static int vimc_cap_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_cap_comp_ops);
-}
-
-static int vimc_cap_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_cap_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_cap_driver_ids[] = {
-	{
-		.name           = VIMC_CAP_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_cap_pdrv = {
-	.probe		= vimc_cap_probe,
-	.remove		= vimc_cap_remove,
-	.id_table	= vimc_cap_driver_ids,
-	.driver		= {
-		.name	= VIMC_CAP_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_cap_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Capture");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
-MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/vimc/vimc-core.c b/drivers/media/platform/vimc/vimc-core.c
index c9b351472272..3bd89cf38a90 100644
--- a/drivers/media/platform/vimc/vimc-core.c
+++ b/drivers/media/platform/vimc/vimc-core.c
@@ -5,7 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
  */
 
-#include <linux/component.h>
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/platform_device.h>
@@ -25,12 +24,6 @@
 	.flags = link_flags,					\
 }
 
-/* Structure which describes individual configuration for each entity */
-struct vimc_ent_config {
-	const char *name;
-	const char *drv;
-};
-
 /* Structure which describes links between entities */
 struct vimc_ent_link {
 	unsigned int src_ent;
@@ -42,7 +35,7 @@ struct vimc_ent_link {
 
 /* Structure which describes the whole topology */
 struct vimc_pipeline_config {
-	const struct vimc_ent_config *ents;
+	struct vimc_ent_config *ents;
 	size_t num_ents;
 	const struct vimc_ent_link *links;
 	size_t num_links;
@@ -52,43 +45,61 @@ struct vimc_pipeline_config {
  * Topology Configuration
  */
 
-static const struct vimc_ent_config ent_config[] = {
+static struct vimc_ent_config ent_config[] = {
 	{
 		.name = "Sensor A",
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Sensor B",
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Debayer A",
 		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
 	},
 	{
 		.name = "Debayer B",
 		.drv = "vimc-debayer",
+		.add = vimc_deb_add,
+		.rm = vimc_deb_rm,
 	},
 	{
 		.name = "Raw Capture 0",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 	{
 		.name = "Raw Capture 1",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 	{
 		.name = "RGB/YUV Input",
 		/* TODO: change this to vimc-input when it is implemented */
 		.drv = "vimc-sensor",
+		.add = vimc_sen_add,
+		.rm = vimc_sen_rm,
 	},
 	{
 		.name = "Scaler",
 		.drv = "vimc-scaler",
+		.add = vimc_sca_add,
+		.rm = vimc_sca_rm,
 	},
 	{
 		.name = "RGB/YUV Capture",
 		.drv = "vimc-capture",
+		.add = vimc_cap_add,
+		.rm = vimc_cap_rm,
 	},
 };
 
@@ -111,7 +122,7 @@ static const struct vimc_ent_link ent_links[] = {
 	VIMC_ENT_LINK(7, 1, 8, 0, MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE),
 };
 
-static const struct vimc_pipeline_config pipe_cfg = {
+static struct vimc_pipeline_config pipe_cfg = {
 	.ents		= ent_config,
 	.num_ents	= ARRAY_SIZE(ent_config),
 	.links		= ent_links,
@@ -128,14 +139,11 @@ static int vimc_create_links(struct vimc_device *vimc)
 	/* Initialize the links between entities */
 	for (i = 0; i < vimc->pipe_cfg->num_links; i++) {
 		const struct vimc_ent_link *link = &vimc->pipe_cfg->links[i];
-		/*
-		 * TODO: Check another way of retrieving ved struct without
-		 * relying on platform_get_drvdata
-		 */
+
 		struct vimc_ent_device *ved_src =
-			platform_get_drvdata(vimc->subdevs[link->src_ent]);
+			vimc->pipe_cfg->ents[link->src_ent].ved;
 		struct vimc_ent_device *ved_sink =
-			platform_get_drvdata(vimc->subdevs[link->sink_ent]);
+			vimc->pipe_cfg->ents[link->sink_ent].ved;
 
 		ret = media_create_pad_link(ved_src->ent, link->src_pad,
 					    ved_sink->ent, link->sink_pad,
@@ -147,13 +155,28 @@ static int vimc_create_links(struct vimc_device *vimc)
 	return 0;
 }
 
-static int vimc_comp_bind(struct device *master)
+static int vimc_add_subdevs(struct vimc_device *vimc)
 {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-	int ret;
+	int i;
+	int ret = 0;
+
+	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
+		dev_dbg(&vimc->pdev.dev, "new entity for %s\n",
+			vimc->pipe_cfg->ents[i].name);
+		ret = vimc->pipe_cfg->ents[i].add(vimc,
+				&vimc->pipe_cfg->ents[i]);
+		if (ret) {
+			dev_err(&vimc->pdev.dev, "add new entity for %s\n",
+				vimc->pipe_cfg->ents[i].name);
+			return ret;
+		}
+	}
+	return 0;
+}
 
-	dev_dbg(master, "bind");
+static int vimc_register_devices(struct vimc_device *vimc)
+{
+	int ret;
 
 	/* Register the v4l2 struct */
 	ret = v4l2_device_register(vimc->mdev.dev, &vimc->v4l2_dev);
@@ -163,22 +186,20 @@ static int vimc_comp_bind(struct device *master)
 		return ret;
 	}
 
-	/* Bind subdevices */
-	ret = component_bind_all(master, &vimc->v4l2_dev);
-	if (ret)
-		goto err_v4l2_unregister;
+	/* Invoke enitity config hooks to initialize and register subdevs */
+	vimc_add_subdevs(vimc);
 
 	/* Initialize links */
 	ret = vimc_create_links(vimc);
 	if (ret)
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 
 	/* Register the media device */
 	ret = media_device_register(&vimc->mdev);
 	if (ret) {
 		dev_err(vimc->mdev.dev,
 			"media device register failed (err=%d)\n", ret);
-		goto err_comp_unbind_all;
+		goto err_v4l2_unregister;
 	}
 
 	/* Expose all subdev's nodes*/
@@ -195,98 +216,36 @@ static int vimc_comp_bind(struct device *master)
 err_mdev_unregister:
 	media_device_unregister(&vimc->mdev);
 	media_device_cleanup(&vimc->mdev);
-err_comp_unbind_all:
-	component_unbind_all(master, NULL);
 err_v4l2_unregister:
 	v4l2_device_unregister(&vimc->v4l2_dev);
 
 	return ret;
 }
 
-static void vimc_comp_unbind(struct device *master)
+static void vimc_unregister(struct vimc_device *vimc)
 {
-	struct vimc_device *vimc = container_of(to_platform_device(master),
-						struct vimc_device, pdev);
-
-	dev_dbg(master, "unbind");
-
 	media_device_unregister(&vimc->mdev);
 	media_device_cleanup(&vimc->mdev);
-	component_unbind_all(master, NULL);
 	v4l2_device_unregister(&vimc->v4l2_dev);
 }
 
-static int vimc_comp_compare(struct device *comp, void *data)
-{
-	return comp == data;
-}
-
-static struct component_match *vimc_add_subdevs(struct vimc_device *vimc)
-{
-	struct component_match *match = NULL;
-	struct vimc_platform_data pdata;
-	int i;
-
-	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
-		dev_dbg(&vimc->pdev.dev, "new pdev for %s\n",
-			vimc->pipe_cfg->ents[i].drv);
-
-		strscpy(pdata.entity_name, vimc->pipe_cfg->ents[i].name,
-			sizeof(pdata.entity_name));
-
-		vimc->subdevs[i] = platform_device_register_data(&vimc->pdev.dev,
-						vimc->pipe_cfg->ents[i].drv,
-						PLATFORM_DEVID_AUTO,
-						&pdata,
-						sizeof(pdata));
-		if (IS_ERR(vimc->subdevs[i])) {
-			match = ERR_CAST(vimc->subdevs[i]);
-			while (--i >= 0)
-				platform_device_unregister(vimc->subdevs[i]);
-
-			return match;
-		}
-
-		component_match_add(&vimc->pdev.dev, &match, vimc_comp_compare,
-				    &vimc->subdevs[i]->dev);
-	}
-
-	return match;
-}
-
 static void vimc_rm_subdevs(struct vimc_device *vimc)
 {
 	unsigned int i;
 
 	for (i = 0; i < vimc->pipe_cfg->num_ents; i++)
-		platform_device_unregister(vimc->subdevs[i]);
+		vimc->pipe_cfg->ents[i].rm(vimc, &vimc->pipe_cfg->ents[i]);
 }
 
-static const struct component_master_ops vimc_comp_ops = {
-	.bind = vimc_comp_bind,
-	.unbind = vimc_comp_unbind,
-};
-
 static int vimc_probe(struct platform_device *pdev)
 {
 	struct vimc_device *vimc = container_of(pdev, struct vimc_device, pdev);
-	struct component_match *match = NULL;
-	int ret;
+	int ret = 0;
 
 	dev_dbg(&pdev->dev, "probe");
 
 	memset(&vimc->mdev, 0, sizeof(vimc->mdev));
 
-	/* Create platform_device for each entity in the topology*/
-	vimc->subdevs = devm_kcalloc(&vimc->pdev.dev, vimc->pipe_cfg->num_ents,
-				     sizeof(*vimc->subdevs), GFP_KERNEL);
-	if (!vimc->subdevs)
-		return -ENOMEM;
-
-	match = vimc_add_subdevs(vimc);
-	if (IS_ERR(match))
-		return PTR_ERR(match);
-
 	/* Link the media device within the v4l2_device */
 	vimc->v4l2_dev.mdev = &vimc->mdev;
 
@@ -298,16 +257,11 @@ static int vimc_probe(struct platform_device *pdev)
 	vimc->mdev.dev = &pdev->dev;
 	media_device_init(&vimc->mdev);
 
-	/* Add self to the component system */
-	ret = component_master_add_with_match(&pdev->dev, &vimc_comp_ops,
-					      match);
-	if (ret) {
+	ret = vimc_register_devices(vimc);
+	if (ret)
 		media_device_cleanup(&vimc->mdev);
-		vimc_rm_subdevs(vimc);
-		return ret;
-	}
 
-	return 0;
+	return ret;
 }
 
 static int vimc_remove(struct platform_device *pdev)
@@ -316,8 +270,8 @@ static int vimc_remove(struct platform_device *pdev)
 
 	dev_dbg(&pdev->dev, "remove");
 
-	component_master_del(&pdev->dev, &vimc_comp_ops);
 	vimc_rm_subdevs(vimc);
+	vimc_unregister(vimc);
 
 	return 0;
 }
diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
index 750752bb173c..b58e0946ac2a 100644
--- a/drivers/media/platform/vimc/vimc-debayer.c
+++ b/drivers/media/platform/vimc/vimc-debayer.c
@@ -5,10 +5,7 @@
  * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
  */
 
-#include <linux/component.h>
 #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/vmalloc.h>
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
@@ -16,9 +13,7 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_DEB_DRV_NAME "vimc-debayer"
-/* This module only supports transforming a bayer format
- * to V4L2_PIX_FMT_RGB24
+/* Supports transforming a bayer format to V4L2_PIX_FMT_RGB24
  */
 #define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
 #define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24
@@ -499,21 +494,19 @@ static const struct v4l2_subdev_internal_ops vimc_deb_int_ops = {
 	.release = vimc_deb_release,
 };
 
-static void vimc_deb_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_deb_device *vdeb = container_of(ved, struct vimc_deb_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_deb_device *vdeb = container_of(ved,
+						    struct vimc_deb_device,
 						    ved);
 
 	vimc_ent_sd_unregister(ved, &vdeb->sd);
 }
 
-static int vimc_deb_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_deb_device *vdeb;
 	int ret;
 
@@ -524,7 +517,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vdeb->ved, &vdeb->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
 				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
 				   MEDIA_PAD_FL_SOURCE},
@@ -535,8 +528,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 	}
 
 	vdeb->ved.process_frame = vimc_deb_process_frame;
-	dev_set_drvdata(comp, &vdeb->ved);
-	vdeb->dev = comp;
+	vdeb->dev = &vimc->pdev.dev;
 
 	/* Initialize the frame format */
 	vdeb->sink_fmt = sink_fmt_default;
@@ -549,46 +541,6 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
 	 */
 	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
 
+	vent->ved = &vdeb->ved;
 	return 0;
 }
-
-static const struct component_ops vimc_deb_comp_ops = {
-	.bind = vimc_deb_comp_bind,
-	.unbind = vimc_deb_comp_unbind,
-};
-
-static int vimc_deb_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_deb_comp_ops);
-}
-
-static int vimc_deb_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_deb_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_deb_driver_ids[] = {
-	{
-		.name           = VIMC_DEB_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_deb_pdrv = {
-	.probe		= vimc_deb_probe,
-	.remove		= vimc_deb_remove,
-	.id_table	= vimc_deb_driver_ids,
-	.driver		= {
-		.name	= VIMC_DEB_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_deb_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Debayer");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
-MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
index fe99b9102ada..7cd478fee3ae 100644
--- a/drivers/media/platform/vimc/vimc-scaler.c
+++ b/drivers/media/platform/vimc/vimc-scaler.c
@@ -5,10 +5,7 @@
  * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
  */
 
-#include <linux/component.h>
 #include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/vmalloc.h>
 #include <linux/v4l2-mediabus.h>
 #include <media/v4l2-subdev.h>
@@ -16,8 +13,6 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_SCA_DRV_NAME "vimc-scaler"
-
 static unsigned int sca_mult = 3;
 module_param(sca_mult, uint, 0000);
 MODULE_PARM_DESC(sca_mult, " the image size multiplier");
@@ -331,22 +326,21 @@ static const struct v4l2_subdev_internal_ops vimc_sca_int_ops = {
 	.release = vimc_sca_release,
 };
 
-static void vimc_sca_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sca_device *vsca = container_of(ved, struct vimc_sca_device,
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sca_device *vsca = container_of(ved,
+						    struct vimc_sca_device,
 						    ved);
 
+
 	vimc_ent_sd_unregister(ved, &vsca->sd);
 }
 
 
-static int vimc_sca_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_sca_device *vsca;
 	int ret;
 
@@ -357,7 +351,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vsca->ved, &vsca->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
 				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
 				   MEDIA_PAD_FL_SOURCE},
@@ -368,52 +362,12 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
 	}
 
 	vsca->ved.process_frame = vimc_sca_process_frame;
-	dev_set_drvdata(comp, &vsca->ved);
-	vsca->dev = comp;
+	vsca->dev = &vimc->pdev.dev;
+
+	vent->ved = &vsca->ved;
 
 	/* Initialize the frame format */
 	vsca->sink_fmt = sink_fmt_default;
 
 	return 0;
 }
-
-static const struct component_ops vimc_sca_comp_ops = {
-	.bind = vimc_sca_comp_bind,
-	.unbind = vimc_sca_comp_unbind,
-};
-
-static int vimc_sca_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sca_comp_ops);
-}
-
-static int vimc_sca_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sca_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sca_driver_ids[] = {
-	{
-		.name           = VIMC_SCA_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sca_pdrv = {
-	.probe		= vimc_sca_probe,
-	.remove		= vimc_sca_remove,
-	.id_table	= vimc_sca_driver_ids,
-	.driver		= {
-		.name	= VIMC_SCA_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sca_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sca_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Scaler");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
-MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
index 6c57b1e262f9..bd237e030a1b 100644
--- a/drivers/media/platform/vimc/vimc-sensor.c
+++ b/drivers/media/platform/vimc/vimc-sensor.c
@@ -5,10 +5,6 @@
  * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
  */
 
-#include <linux/component.h>
-#include <linux/module.h>
-#include <linux/mod_devicetable.h>
-#include <linux/platform_device.h>
 #include <linux/v4l2-mediabus.h>
 #include <linux/vmalloc.h>
 #include <media/v4l2-ctrls.h>
@@ -19,8 +15,6 @@
 #include "vimc.h"
 #include "vimc-common.h"
 
-#define VIMC_SEN_DRV_NAME "vimc-sensor"
-
 static const struct v4l2_mbus_framefmt fmt_default = {
 	.width = 640,
 	.height = 480,
@@ -268,12 +262,12 @@ static const struct v4l2_subdev_internal_ops vimc_sen_int_ops = {
 	.release = vimc_sen_release,
 };
 
-static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
-				 void *master_data)
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct vimc_ent_device *ved = dev_get_drvdata(comp);
-	struct vimc_sen_device *vsen =
-				container_of(ved, struct vimc_sen_device, ved);
+	struct vimc_ent_device *ved = vent->ved;
+	struct vimc_sen_device *vsen = container_of(ved,
+						    struct vimc_sen_device,
+						    ved);
 
 	vimc_ent_sd_unregister(ved, &vsen->sd);
 }
@@ -295,11 +289,9 @@ static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
 	.qmenu = tpg_pattern_strings,
 };
 
-static int vimc_sen_comp_bind(struct device *comp, struct device *master,
-			      void *master_data)
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
 {
-	struct v4l2_device *v4l2_dev = master_data;
-	struct vimc_platform_data *pdata = comp->platform_data;
+	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
 	struct vimc_sen_device *vsen;
 	int ret;
 
@@ -332,7 +324,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
 	/* Initialize ved and sd */
 	ret = vimc_ent_sd_register(&vsen->ved, &vsen->sd, v4l2_dev,
-				   pdata->entity_name,
+				   vent->name,
 				   MEDIA_ENT_F_CAM_SENSOR, 1,
 				   (const unsigned long[1]) {MEDIA_PAD_FL_SOURCE},
 				   &vimc_sen_int_ops, &vimc_sen_ops);
@@ -340,8 +332,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 		goto err_free_hdl;
 
 	vsen->ved.process_frame = vimc_sen_process_frame;
-	dev_set_drvdata(comp, &vsen->ved);
-	vsen->dev = comp;
+	vsen->dev = &vimc->pdev.dev;
 
 	/* Initialize the frame format */
 	vsen->mbus_format = fmt_default;
@@ -353,6 +344,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 	if (ret)
 		goto err_unregister_ent_sd;
 
+	vent->ved = &vsen->ved;
 	return 0;
 
 err_unregister_ent_sd:
@@ -364,44 +356,3 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
 
 	return ret;
 }
-
-static const struct component_ops vimc_sen_comp_ops = {
-	.bind = vimc_sen_comp_bind,
-	.unbind = vimc_sen_comp_unbind,
-};
-
-static int vimc_sen_probe(struct platform_device *pdev)
-{
-	return component_add(&pdev->dev, &vimc_sen_comp_ops);
-}
-
-static int vimc_sen_remove(struct platform_device *pdev)
-{
-	component_del(&pdev->dev, &vimc_sen_comp_ops);
-
-	return 0;
-}
-
-static const struct platform_device_id vimc_sen_driver_ids[] = {
-	{
-		.name           = VIMC_SEN_DRV_NAME,
-	},
-	{ }
-};
-
-static struct platform_driver vimc_sen_pdrv = {
-	.probe		= vimc_sen_probe,
-	.remove		= vimc_sen_remove,
-	.id_table	= vimc_sen_driver_ids,
-	.driver		= {
-		.name	= VIMC_SEN_DRV_NAME,
-	},
-};
-
-module_platform_driver(vimc_sen_pdrv);
-
-MODULE_DEVICE_TABLE(platform, vimc_sen_driver_ids);
-
-MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Sensor");
-MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
-MODULE_LICENSE("GPL");
diff --git a/drivers/media/platform/vimc/vimc.h b/drivers/media/platform/vimc/vimc.h
index a5adebdda941..cd527c6fe6fc 100644
--- a/drivers/media/platform/vimc/vimc.h
+++ b/drivers/media/platform/vimc/vimc.h
@@ -92,11 +92,30 @@ struct vimc_device {
 	/* The Associated media_device parent */
 	struct media_device mdev;
 
-	/* Internal v4l2 parent device*/
+	/* Internal v4l2 parent device */
 	struct v4l2_device v4l2_dev;
+};
 
-	/* Subdevices */
-	struct platform_device **subdevs;
+/* Structure which describes individual configuration for each entity */
+struct vimc_ent_config {
+	const char *name;
+	const char *drv;
+	struct vimc_ent_device *ved;
+	int (*add)(struct vimc_device *vimc, struct vimc_ent_config *vent);
+	void (*rm)(struct vimc_device *vimc, struct vimc_ent_config *vent);
 };
 
+/* prototypes for vimc_ent_config add and rm hooks */
+int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
+int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
+void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
+
 #endif
-- 
2.17.1

Re: [PATCH 2/3] media: vimc: Collapse component structure into a single monolithic driver

[Helen Koike]

Hi Shuah,

Thanks for the patch, just some small comments.

On 8/9/19 6:45 PM, Shuah Khan wrote:
> vimc uses Component API to split the driver into functional components.
> The real hardware resembles a monolith structure than component and
> component structure added a level of complexity making it hard to
> maintain without adding any real benefit.
> 
> The sensor is one vimc component that would makes sense to be a separate
> module to closely align with the real hardware. It would be easier to
> collapse vimc into single monolithic driver first and then split the
> sensor off as a separate module.
> 
> Collapse it into a single monolithic driver removing the Component API.
> This patch removes the component API and makes minimal changes to the
> code base preserving the functional division of the code structure.
> Preserving the functional structure allows us to split the sensor off
> as a separate module in the future.
> 
> Major design elements in this change are:
> - Use existing struct vimc_ent_config and struct vimc_pipeline_config
>   to drive the initialization of the functional components.
> - Make vimc_ent_config global by moving it to vimc.h
> - Add two new hooks add and rm to initialize and register, unregister
>   and free subdevs.
> - All component API is now gone and bind and unbind hooks are modified
>   to do "add" and "rm" with minimal changes to just add and rm subdevs.
> - vimc-core's bind and unbind are now register and unregister.
> - vimc-core invokes "add" hooks from its vimc_register_devices().
>   The "add" hooks remain the same and register subdevs. They don't
>   create platform devices of their own and use vimc's pdev.dev as
>   their reference device. The "add" hooks save their vimc_ent_device(s)
>   in the corresponding vimc_ent_config.
> - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
>   and cleanup.
> - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
>   and the corresponding struct vimc_ent_config pointer.
> 
> The following configure and stream test works on all devices.
> 
> media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
> media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
> media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
> media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
> 
> v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
> v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
> v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
> 
> v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
> v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
> v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
> 
> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
> ---
>  drivers/media/platform/vimc/Makefile       |   7 +-
>  drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
>  drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
>  drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
>  drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
>  drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
>  drivers/media/platform/vimc/vimc.h         |  25 +++-
>  7 files changed, 125 insertions(+), 343 deletions(-)
> 
> diff --git a/drivers/media/platform/vimc/Makefile b/drivers/media/platform/vimc/Makefile
> index 96d06f030c31..a53b2b532e9f 100644
> --- a/drivers/media/platform/vimc/Makefile
> +++ b/drivers/media/platform/vimc/Makefile
> @@ -1,5 +1,6 @@
>  # SPDX-License-Identifier: GPL-2.0
> -vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
> +vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
> +		vimc-debayer.o vimc-scaler.o vimc-sensor.o
> +
> +obj-$(CONFIG_VIDEO_VIMC) += vimc.o
>  
> -obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
> -                vimc-scaler.o vimc-sensor.o
> diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
> index c52fc5d97c2d..b7b2d3c3d4f8 100644
> --- a/drivers/media/platform/vimc/vimc-capture.c
> +++ b/drivers/media/platform/vimc/vimc-capture.c
> @@ -5,10 +5,6 @@
>   * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>   */
>  
> -#include <linux/component.h>
> -#include <linux/module.h>
> -#include <linux/mod_devicetable.h>
> -#include <linux/platform_device.h>
>  #include <media/v4l2-ioctl.h>
>  #include <media/videobuf2-core.h>
>  #include <media/videobuf2-vmalloc.h>
> @@ -17,8 +13,6 @@
>  #include "vimc-common.h"
>  #include "vimc-streamer.h"
>  
> -#define VIMC_CAP_DRV_NAME "vimc-capture"
> -
>  static const u32 vimc_cap_supported_pixfmt[] = {
>  	V4L2_PIX_FMT_BGR24,
>  	V4L2_PIX_FMT_RGB24,
> @@ -348,11 +342,11 @@ static void vimc_cap_release(struct video_device *vdev)
>  	kfree(vcap);
>  }
>  
> -static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
> -				 void *master_data)
> +void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
> -	struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
> +	struct vimc_ent_device *ved = vent->ved;
> +	struct vimc_cap_device *vcap = container_of(ved,
> +						    struct vimc_cap_device,
>  						    ved);
>  
>  	vb2_queue_release(&vcap->queue);
> @@ -399,11 +393,9 @@ static void *vimc_cap_process_frame(struct vimc_ent_device *ved,
>  	return NULL;
>  }
>  
> -static int vimc_cap_comp_bind(struct device *comp, struct device *master,
> -			      void *master_data)
> +int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct v4l2_device *v4l2_dev = master_data;
> -	struct vimc_platform_data *pdata = comp->platform_data;
> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>  	struct vimc_cap_device *vcap;
>  	struct video_device *vdev;
>  	struct vb2_queue *q;
> @@ -423,7 +415,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>  	}
>  
>  	/* Initialize the media entity */
> -	vcap->vdev.entity.name = pdata->entity_name;
> +	vcap->vdev.entity.name = vent->name;
>  	vcap->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
>  	ret = media_entity_pads_init(&vcap->vdev.entity,
>  				     1, vcap->ved.pads);
> @@ -447,8 +439,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>  
>  	ret = vb2_queue_init(q);
>  	if (ret) {
> -		dev_err(comp, "%s: vb2 queue init failed (err=%d)\n",
> -			pdata->entity_name, ret);
> +		dev_err(&vimc->pdev.dev, "%s: vb2 queue init failed (err=%d)\n",
> +			vent->name, ret);
>  		goto err_clean_m_ent;
>  	}
>  
> @@ -465,8 +457,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>  	vcap->ved.ent = &vcap->vdev.entity;
>  	vcap->ved.process_frame = vimc_cap_process_frame;
>  	vcap->ved.vdev_get_format = vimc_cap_get_format;
> -	dev_set_drvdata(comp, &vcap->ved);
> -	vcap->dev = comp;
> +	vcap->dev = &vimc->pdev.dev;
>  
>  	/* Initialize the video_device struct */
>  	vdev = &vcap->vdev;
> @@ -479,17 +470,18 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>  	vdev->queue = q;
>  	vdev->v4l2_dev = v4l2_dev;
>  	vdev->vfl_dir = VFL_DIR_RX;
> -	strscpy(vdev->name, pdata->entity_name, sizeof(vdev->name));
> +	strscpy(vdev->name, vent->name, sizeof(vdev->name));
>  	video_set_drvdata(vdev, &vcap->ved);
>  
>  	/* Register the video_device with the v4l2 and the media framework */
>  	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
>  	if (ret) {
> -		dev_err(comp, "%s: video register failed (err=%d)\n",
> +		dev_err(&vimc->pdev.dev, "%s: video register failed (err=%d)\n",
>  			vcap->vdev.name, ret);
>  		goto err_release_queue;
>  	}
>  
> +	vent->ved = &vcap->ved;
>  	return 0;
>  
>  err_release_queue:
> @@ -503,44 +495,3 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>  
>  	return ret;
>  }
> -
> -static const struct component_ops vimc_cap_comp_ops = {
> -	.bind = vimc_cap_comp_bind,
> -	.unbind = vimc_cap_comp_unbind,
> -};
> -
> -static int vimc_cap_probe(struct platform_device *pdev)
> -{
> -	return component_add(&pdev->dev, &vimc_cap_comp_ops);
> -}
> -
> -static int vimc_cap_remove(struct platform_device *pdev)
> -{
> -	component_del(&pdev->dev, &vimc_cap_comp_ops);
> -
> -	return 0;
> -}
> -
> -static const struct platform_device_id vimc_cap_driver_ids[] = {
> -	{
> -		.name           = VIMC_CAP_DRV_NAME,
> -	},
> -	{ }
> -};
> -
> -static struct platform_driver vimc_cap_pdrv = {
> -	.probe		= vimc_cap_probe,
> -	.remove		= vimc_cap_remove,
> -	.id_table	= vimc_cap_driver_ids,
> -	.driver		= {
> -		.name	= VIMC_CAP_DRV_NAME,
> -	},
> -};
> -
> -module_platform_driver(vimc_cap_pdrv);
> -
> -MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);
> -
> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Capture");
> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
> -MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/vimc/vimc-core.c b/drivers/media/platform/vimc/vimc-core.c
> index c9b351472272..3bd89cf38a90 100644
> --- a/drivers/media/platform/vimc/vimc-core.c
> +++ b/drivers/media/platform/vimc/vimc-core.c
> @@ -5,7 +5,6 @@
>   * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>   */
>  
> -#include <linux/component.h>
>  #include <linux/init.h>
>  #include <linux/module.h>
>  #include <linux/platform_device.h>
> @@ -25,12 +24,6 @@
>  	.flags = link_flags,					\
>  }
>  
> -/* Structure which describes individual configuration for each entity */
> -struct vimc_ent_config {
> -	const char *name;
> -	const char *drv;
> -};
> -
>  /* Structure which describes links between entities */
>  struct vimc_ent_link {
>  	unsigned int src_ent;
> @@ -42,7 +35,7 @@ struct vimc_ent_link {
>  
>  /* Structure which describes the whole topology */
>  struct vimc_pipeline_config {
> -	const struct vimc_ent_config *ents;
> +	struct vimc_ent_config *ents;
>  	size_t num_ents;
>  	const struct vimc_ent_link *links;
>  	size_t num_links;
> @@ -52,43 +45,61 @@ struct vimc_pipeline_config {
>   * Topology Configuration
>   */
>  
> -static const struct vimc_ent_config ent_config[] = {
> +static struct vimc_ent_config ent_config[] = {
>  	{
>  		.name = "Sensor A",
>  		.drv = "vimc-sensor",
> +		.add = vimc_sen_add,
> +		.rm = vimc_sen_rm,
>  	},
>  	{
>  		.name = "Sensor B",
>  		.drv = "vimc-sensor",
> +		.add = vimc_sen_add,
> +		.rm = vimc_sen_rm,
>  	},
>  	{
>  		.name = "Debayer A",
>  		.drv = "vimc-debayer",
> +		.add = vimc_deb_add,
> +		.rm = vimc_deb_rm,
>  	},
>  	{
>  		.name = "Debayer B",
>  		.drv = "vimc-debayer",
> +		.add = vimc_deb_add,
> +		.rm = vimc_deb_rm,
>  	},
>  	{
>  		.name = "Raw Capture 0",
>  		.drv = "vimc-capture",
> +		.add = vimc_cap_add,
> +		.rm = vimc_cap_rm,
>  	},
>  	{
>  		.name = "Raw Capture 1",
>  		.drv = "vimc-capture",
> +		.add = vimc_cap_add,
> +		.rm = vimc_cap_rm,
>  	},
>  	{
>  		.name = "RGB/YUV Input",
>  		/* TODO: change this to vimc-input when it is implemented */
>  		.drv = "vimc-sensor",
> +		.add = vimc_sen_add,
> +		.rm = vimc_sen_rm,
>  	},
>  	{
>  		.name = "Scaler",
>  		.drv = "vimc-scaler",
> +		.add = vimc_sca_add,
> +		.rm = vimc_sca_rm,
>  	},
>  	{
>  		.name = "RGB/YUV Capture",
>  		.drv = "vimc-capture",
> +		.add = vimc_cap_add,
> +		.rm = vimc_cap_rm,
>  	},
>  };
>  
> @@ -111,7 +122,7 @@ static const struct vimc_ent_link ent_links[] = {
>  	VIMC_ENT_LINK(7, 1, 8, 0, MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE),
>  };
>  
> -static const struct vimc_pipeline_config pipe_cfg = {
> +static struct vimc_pipeline_config pipe_cfg = {

Any special reason why the const was dropped here?

>  	.ents		= ent_config,
>  	.num_ents	= ARRAY_SIZE(ent_config),
>  	.links		= ent_links,
> @@ -128,14 +139,11 @@ static int vimc_create_links(struct vimc_device *vimc)
>  	/* Initialize the links between entities */
>  	for (i = 0; i < vimc->pipe_cfg->num_links; i++) {
>  		const struct vimc_ent_link *link = &vimc->pipe_cfg->links[i];
> -		/*
> -		 * TODO: Check another way of retrieving ved struct without
> -		 * relying on platform_get_drvdata
> -		 */
> +
>  		struct vimc_ent_device *ved_src =
> -			platform_get_drvdata(vimc->subdevs[link->src_ent]);
> +			vimc->pipe_cfg->ents[link->src_ent].ved;
>  		struct vimc_ent_device *ved_sink =
> -			platform_get_drvdata(vimc->subdevs[link->sink_ent]);
> +			vimc->pipe_cfg->ents[link->sink_ent].ved;
>  
>  		ret = media_create_pad_link(ved_src->ent, link->src_pad,
>  					    ved_sink->ent, link->sink_pad,
> @@ -147,13 +155,28 @@ static int vimc_create_links(struct vimc_device *vimc)
>  	return 0;
>  }
>  
> -static int vimc_comp_bind(struct device *master)
> +static int vimc_add_subdevs(struct vimc_device *vimc)
>  {
> -	struct vimc_device *vimc = container_of(to_platform_device(master),
> -						struct vimc_device, pdev);
> -	int ret;
> +	int i;

unsigned int

> +	int ret = 0;

no need to init to 0, but not a problem either.

> +
> +	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
> +		dev_dbg(&vimc->pdev.dev, "new entity for %s\n",
> +			vimc->pipe_cfg->ents[i].name);
> +		ret = vimc->pipe_cfg->ents[i].add(vimc,
> +				&vimc->pipe_cfg->ents[i]);
> +		if (ret) {
> +			dev_err(&vimc->pdev.dev, "add new entity for %s\n",
> +				vimc->pipe_cfg->ents[i].name);
> +			return ret;
> +		}
> +	}
> +	return 0;
> +}
>  
> -	dev_dbg(master, "bind");
> +static int vimc_register_devices(struct vimc_device *vimc)
> +{
> +	int ret;
>  
>  	/* Register the v4l2 struct */
>  	ret = v4l2_device_register(vimc->mdev.dev, &vimc->v4l2_dev);
> @@ -163,22 +186,20 @@ static int vimc_comp_bind(struct device *master)
>  		return ret;
>  	}
>  
> -	/* Bind subdevices */
> -	ret = component_bind_all(master, &vimc->v4l2_dev);
> -	if (ret)
> -		goto err_v4l2_unregister;
> +	/* Invoke enitity config hooks to initialize and register subdevs */
> +	vimc_add_subdevs(vimc);
>  
>  	/* Initialize links */
>  	ret = vimc_create_links(vimc);
>  	if (ret)
> -		goto err_comp_unbind_all;
> +		goto err_v4l2_unregister;
>  
>  	/* Register the media device */
>  	ret = media_device_register(&vimc->mdev);
>  	if (ret) {
>  		dev_err(vimc->mdev.dev,
>  			"media device register failed (err=%d)\n", ret);
> -		goto err_comp_unbind_all;
> +		goto err_v4l2_unregister;
>  	}
>  
>  	/* Expose all subdev's nodes*/
> @@ -195,98 +216,36 @@ static int vimc_comp_bind(struct device *master)
>  err_mdev_unregister:
>  	media_device_unregister(&vimc->mdev);
>  	media_device_cleanup(&vimc->mdev);
> -err_comp_unbind_all:
> -	component_unbind_all(master, NULL);
>  err_v4l2_unregister:
>  	v4l2_device_unregister(&vimc->v4l2_dev);
>  
>  	return ret;
>  }
>  
> -static void vimc_comp_unbind(struct device *master)
> +static void vimc_unregister(struct vimc_device *vimc)
>  {
> -	struct vimc_device *vimc = container_of(to_platform_device(master),
> -						struct vimc_device, pdev);
> -
> -	dev_dbg(master, "unbind");
> -
>  	media_device_unregister(&vimc->mdev);
>  	media_device_cleanup(&vimc->mdev);
> -	component_unbind_all(master, NULL);
>  	v4l2_device_unregister(&vimc->v4l2_dev);
>  }
>  
> -static int vimc_comp_compare(struct device *comp, void *data)
> -{
> -	return comp == data;
> -}
> -
> -static struct component_match *vimc_add_subdevs(struct vimc_device *vimc)
> -{
> -	struct component_match *match = NULL;
> -	struct vimc_platform_data pdata;
> -	int i;
> -
> -	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
> -		dev_dbg(&vimc->pdev.dev, "new pdev for %s\n",
> -			vimc->pipe_cfg->ents[i].drv);
> -
> -		strscpy(pdata.entity_name, vimc->pipe_cfg->ents[i].name,
> -			sizeof(pdata.entity_name));
> -
> -		vimc->subdevs[i] = platform_device_register_data(&vimc->pdev.dev,
> -						vimc->pipe_cfg->ents[i].drv,
> -						PLATFORM_DEVID_AUTO,
> -						&pdata,
> -						sizeof(pdata));
> -		if (IS_ERR(vimc->subdevs[i])) {
> -			match = ERR_CAST(vimc->subdevs[i]);
> -			while (--i >= 0)
> -				platform_device_unregister(vimc->subdevs[i]);
> -
> -			return match;
> -		}
> -
> -		component_match_add(&vimc->pdev.dev, &match, vimc_comp_compare,
> -				    &vimc->subdevs[i]->dev);
> -	}
> -
> -	return match;
> -}
> -
>  static void vimc_rm_subdevs(struct vimc_device *vimc)
>  {
>  	unsigned int i;
>  
>  	for (i = 0; i < vimc->pipe_cfg->num_ents; i++)
> -		platform_device_unregister(vimc->subdevs[i]);
> +		vimc->pipe_cfg->ents[i].rm(vimc, &vimc->pipe_cfg->ents[i]);
>  }
>  
> -static const struct component_master_ops vimc_comp_ops = {
> -	.bind = vimc_comp_bind,
> -	.unbind = vimc_comp_unbind,
> -};
> -
>  static int vimc_probe(struct platform_device *pdev)
>  {
>  	struct vimc_device *vimc = container_of(pdev, struct vimc_device, pdev);
> -	struct component_match *match = NULL;
> -	int ret;
> +	int ret = 0;
>  
>  	dev_dbg(&pdev->dev, "probe");
>  
>  	memset(&vimc->mdev, 0, sizeof(vimc->mdev));
>  
> -	/* Create platform_device for each entity in the topology*/
> -	vimc->subdevs = devm_kcalloc(&vimc->pdev.dev, vimc->pipe_cfg->num_ents,
> -				     sizeof(*vimc->subdevs), GFP_KERNEL);
> -	if (!vimc->subdevs)
> -		return -ENOMEM;
> -
> -	match = vimc_add_subdevs(vimc);
> -	if (IS_ERR(match))
> -		return PTR_ERR(match);
> -
>  	/* Link the media device within the v4l2_device */
>  	vimc->v4l2_dev.mdev = &vimc->mdev;
>  
> @@ -298,16 +257,11 @@ static int vimc_probe(struct platform_device *pdev)
>  	vimc->mdev.dev = &pdev->dev;
>  	media_device_init(&vimc->mdev);
>  
> -	/* Add self to the component system */
> -	ret = component_master_add_with_match(&pdev->dev, &vimc_comp_ops,
> -					      match);
> -	if (ret) {
> +	ret = vimc_register_devices(vimc);
> +	if (ret)
>  		media_device_cleanup(&vimc->mdev);
> -		vimc_rm_subdevs(vimc);
> -		return ret;
> -	}
>  
> -	return 0;
> +	return ret;
>  }
>  
>  static int vimc_remove(struct platform_device *pdev)
> @@ -316,8 +270,8 @@ static int vimc_remove(struct platform_device *pdev)
>  
>  	dev_dbg(&pdev->dev, "remove");
>  
> -	component_master_del(&pdev->dev, &vimc_comp_ops);
>  	vimc_rm_subdevs(vimc);
> +	vimc_unregister(vimc);
>  
>  	return 0;
>  }
> diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
> index 750752bb173c..b58e0946ac2a 100644
> --- a/drivers/media/platform/vimc/vimc-debayer.c
> +++ b/drivers/media/platform/vimc/vimc-debayer.c
> @@ -5,10 +5,7 @@
>   * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>   */
>  
> -#include <linux/component.h>
>  #include <linux/module.h>
> -#include <linux/mod_devicetable.h>
> -#include <linux/platform_device.h>
>  #include <linux/vmalloc.h>
>  #include <linux/v4l2-mediabus.h>
>  #include <media/v4l2-subdev.h>
> @@ -16,9 +13,7 @@
>  #include "vimc.h"
>  #include "vimc-common.h"
>  
> -#define VIMC_DEB_DRV_NAME "vimc-debayer"
> -/* This module only supports transforming a bayer format
> - * to V4L2_PIX_FMT_RGB24
> +/* Supports transforming a bayer format to V4L2_PIX_FMT_RGB24
>   */
>  #define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
>  #define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24
> @@ -499,21 +494,19 @@ static const struct v4l2_subdev_internal_ops vimc_deb_int_ops = {
>  	.release = vimc_deb_release,
>  };
>  
> -static void vimc_deb_comp_unbind(struct device *comp, struct device *master,
> -				 void *master_data)
> +void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
> -	struct vimc_deb_device *vdeb = container_of(ved, struct vimc_deb_device,
> +	struct vimc_ent_device *ved = vent->ved;
> +	struct vimc_deb_device *vdeb = container_of(ved,
> +						    struct vimc_deb_device,
>  						    ved);
>  
>  	vimc_ent_sd_unregister(ved, &vdeb->sd);
>  }
>  
> -static int vimc_deb_comp_bind(struct device *comp, struct device *master,
> -			      void *master_data)
> +int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct v4l2_device *v4l2_dev = master_data;
> -	struct vimc_platform_data *pdata = comp->platform_data;
> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>  	struct vimc_deb_device *vdeb;
>  	int ret;
>  
> @@ -524,7 +517,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>  
>  	/* Initialize ved and sd */
>  	ret = vimc_ent_sd_register(&vdeb->ved, &vdeb->sd, v4l2_dev,
> -				   pdata->entity_name,
> +				   vent->name,
>  				   MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
>  				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
>  				   MEDIA_PAD_FL_SOURCE},
> @@ -535,8 +528,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>  	}
>  
>  	vdeb->ved.process_frame = vimc_deb_process_frame;
> -	dev_set_drvdata(comp, &vdeb->ved);
> -	vdeb->dev = comp;
> +	vdeb->dev = &vimc->pdev.dev;
>  
>  	/* Initialize the frame format */
>  	vdeb->sink_fmt = sink_fmt_default;
> @@ -549,46 +541,6 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>  	 */
>  	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
>  
> +	vent->ved = &vdeb->ved;
>  	return 0;
>  }
> -
> -static const struct component_ops vimc_deb_comp_ops = {
> -	.bind = vimc_deb_comp_bind,
> -	.unbind = vimc_deb_comp_unbind,
> -};
> -
> -static int vimc_deb_probe(struct platform_device *pdev)
> -{
> -	return component_add(&pdev->dev, &vimc_deb_comp_ops);
> -}
> -
> -static int vimc_deb_remove(struct platform_device *pdev)
> -{
> -	component_del(&pdev->dev, &vimc_deb_comp_ops);
> -
> -	return 0;
> -}
> -
> -static const struct platform_device_id vimc_deb_driver_ids[] = {
> -	{
> -		.name           = VIMC_DEB_DRV_NAME,
> -	},
> -	{ }
> -};
> -
> -static struct platform_driver vimc_deb_pdrv = {
> -	.probe		= vimc_deb_probe,
> -	.remove		= vimc_deb_remove,
> -	.id_table	= vimc_deb_driver_ids,
> -	.driver		= {
> -		.name	= VIMC_DEB_DRV_NAME,
> -	},
> -};
> -
> -module_platform_driver(vimc_deb_pdrv);
> -
> -MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);
> -
> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Debayer");
> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
> -MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
> index fe99b9102ada..7cd478fee3ae 100644
> --- a/drivers/media/platform/vimc/vimc-scaler.c
> +++ b/drivers/media/platform/vimc/vimc-scaler.c
> @@ -5,10 +5,7 @@
>   * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>   */
>  
> -#include <linux/component.h>
>  #include <linux/module.h>
> -#include <linux/mod_devicetable.h>
> -#include <linux/platform_device.h>
>  #include <linux/vmalloc.h>
>  #include <linux/v4l2-mediabus.h>
>  #include <media/v4l2-subdev.h>
> @@ -16,8 +13,6 @@
>  #include "vimc.h"
>  #include "vimc-common.h"
>  
> -#define VIMC_SCA_DRV_NAME "vimc-scaler"
> -
>  static unsigned int sca_mult = 3;
>  module_param(sca_mult, uint, 0000);
>  MODULE_PARM_DESC(sca_mult, " the image size multiplier");
> @@ -331,22 +326,21 @@ static const struct v4l2_subdev_internal_ops vimc_sca_int_ops = {
>  	.release = vimc_sca_release,
>  };
>  
> -static void vimc_sca_comp_unbind(struct device *comp, struct device *master,
> -				 void *master_data)
> +void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
> -	struct vimc_sca_device *vsca = container_of(ved, struct vimc_sca_device,
> +	struct vimc_ent_device *ved = vent->ved;
> +	struct vimc_sca_device *vsca = container_of(ved,
> +						    struct vimc_sca_device,
>  						    ved);
>  
> +
>  	vimc_ent_sd_unregister(ved, &vsca->sd);
>  }
>  
>  
> -static int vimc_sca_comp_bind(struct device *comp, struct device *master,
> -			      void *master_data)
> +int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct v4l2_device *v4l2_dev = master_data;
> -	struct vimc_platform_data *pdata = comp->platform_data;
> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>  	struct vimc_sca_device *vsca;
>  	int ret;
>  
> @@ -357,7 +351,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
>  
>  	/* Initialize ved and sd */
>  	ret = vimc_ent_sd_register(&vsca->ved, &vsca->sd, v4l2_dev,
> -				   pdata->entity_name,
> +				   vent->name,
>  				   MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
>  				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
>  				   MEDIA_PAD_FL_SOURCE},
> @@ -368,52 +362,12 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
>  	}
>  
>  	vsca->ved.process_frame = vimc_sca_process_frame;
> -	dev_set_drvdata(comp, &vsca->ved);
> -	vsca->dev = comp;
> +	vsca->dev = &vimc->pdev.dev;
> +
> +	vent->ved = &vsca->ved;
>  
>  	/* Initialize the frame format */
>  	vsca->sink_fmt = sink_fmt_default;
>  
>  	return 0;
>  }
> -
> -static const struct component_ops vimc_sca_comp_ops = {
> -	.bind = vimc_sca_comp_bind,
> -	.unbind = vimc_sca_comp_unbind,
> -};
> -
> -static int vimc_sca_probe(struct platform_device *pdev)
> -{
> -	return component_add(&pdev->dev, &vimc_sca_comp_ops);
> -}
> -
> -static int vimc_sca_remove(struct platform_device *pdev)
> -{
> -	component_del(&pdev->dev, &vimc_sca_comp_ops);
> -
> -	return 0;
> -}
> -
> -static const struct platform_device_id vimc_sca_driver_ids[] = {
> -	{
> -		.name           = VIMC_SCA_DRV_NAME,
> -	},
> -	{ }
> -};
> -
> -static struct platform_driver vimc_sca_pdrv = {
> -	.probe		= vimc_sca_probe,
> -	.remove		= vimc_sca_remove,
> -	.id_table	= vimc_sca_driver_ids,
> -	.driver		= {
> -		.name	= VIMC_SCA_DRV_NAME,
> -	},
> -};
> -
> -module_platform_driver(vimc_sca_pdrv);
> -
> -MODULE_DEVICE_TABLE(platform, vimc_sca_driver_ids);
> -
> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Scaler");
> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
> -MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
> index 6c57b1e262f9..bd237e030a1b 100644
> --- a/drivers/media/platform/vimc/vimc-sensor.c
> +++ b/drivers/media/platform/vimc/vimc-sensor.c
> @@ -5,10 +5,6 @@
>   * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>   */
>  
> -#include <linux/component.h>
> -#include <linux/module.h>
> -#include <linux/mod_devicetable.h>
> -#include <linux/platform_device.h>
>  #include <linux/v4l2-mediabus.h>
>  #include <linux/vmalloc.h>
>  #include <media/v4l2-ctrls.h>
> @@ -19,8 +15,6 @@
>  #include "vimc.h"
>  #include "vimc-common.h"
>  
> -#define VIMC_SEN_DRV_NAME "vimc-sensor"
> -
>  static const struct v4l2_mbus_framefmt fmt_default = {
>  	.width = 640,
>  	.height = 480,
> @@ -268,12 +262,12 @@ static const struct v4l2_subdev_internal_ops vimc_sen_int_ops = {
>  	.release = vimc_sen_release,
>  };
>  
> -static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
> -				 void *master_data)
> +void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
> -	struct vimc_sen_device *vsen =
> -				container_of(ved, struct vimc_sen_device, ved);
> +	struct vimc_ent_device *ved = vent->ved;
> +	struct vimc_sen_device *vsen = container_of(ved,
> +						    struct vimc_sen_device,
> +						    ved);
>  
>  	vimc_ent_sd_unregister(ved, &vsen->sd);
>  }
> @@ -295,11 +289,9 @@ static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
>  	.qmenu = tpg_pattern_strings,
>  };
>  
> -static int vimc_sen_comp_bind(struct device *comp, struct device *master,
> -			      void *master_data)
> +int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>  {
> -	struct v4l2_device *v4l2_dev = master_data;
> -	struct vimc_platform_data *pdata = comp->platform_data;
> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>  	struct vimc_sen_device *vsen;
>  	int ret;
>  
> @@ -332,7 +324,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>  
>  	/* Initialize ved and sd */
>  	ret = vimc_ent_sd_register(&vsen->ved, &vsen->sd, v4l2_dev,
> -				   pdata->entity_name,
> +				   vent->name,
>  				   MEDIA_ENT_F_CAM_SENSOR, 1,
>  				   (const unsigned long[1]) {MEDIA_PAD_FL_SOURCE},
>  				   &vimc_sen_int_ops, &vimc_sen_ops);
> @@ -340,8 +332,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>  		goto err_free_hdl;
>  
>  	vsen->ved.process_frame = vimc_sen_process_frame;
> -	dev_set_drvdata(comp, &vsen->ved);
> -	vsen->dev = comp;
> +	vsen->dev = &vimc->pdev.dev;
>  
>  	/* Initialize the frame format */
>  	vsen->mbus_format = fmt_default;
> @@ -353,6 +344,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>  	if (ret)
>  		goto err_unregister_ent_sd;
>  
> +	vent->ved = &vsen->ved;
>  	return 0;
>  
>  err_unregister_ent_sd:
> @@ -364,44 +356,3 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>  
>  	return ret;
>  }
> -
> -static const struct component_ops vimc_sen_comp_ops = {
> -	.bind = vimc_sen_comp_bind,
> -	.unbind = vimc_sen_comp_unbind,
> -};
> -
> -static int vimc_sen_probe(struct platform_device *pdev)
> -{
> -	return component_add(&pdev->dev, &vimc_sen_comp_ops);
> -}
> -
> -static int vimc_sen_remove(struct platform_device *pdev)
> -{
> -	component_del(&pdev->dev, &vimc_sen_comp_ops);
> -
> -	return 0;
> -}
> -
> -static const struct platform_device_id vimc_sen_driver_ids[] = {
> -	{
> -		.name           = VIMC_SEN_DRV_NAME,
> -	},
> -	{ }
> -};
> -
> -static struct platform_driver vimc_sen_pdrv = {
> -	.probe		= vimc_sen_probe,
> -	.remove		= vimc_sen_remove,
> -	.id_table	= vimc_sen_driver_ids,
> -	.driver		= {
> -		.name	= VIMC_SEN_DRV_NAME,
> -	},
> -};
> -
> -module_platform_driver(vimc_sen_pdrv);
> -
> -MODULE_DEVICE_TABLE(platform, vimc_sen_driver_ids);
> -
> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Sensor");
> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
> -MODULE_LICENSE("GPL");
> diff --git a/drivers/media/platform/vimc/vimc.h b/drivers/media/platform/vimc/vimc.h
> index a5adebdda941..cd527c6fe6fc 100644
> --- a/drivers/media/platform/vimc/vimc.h
> +++ b/drivers/media/platform/vimc/vimc.h
> @@ -92,11 +92,30 @@ struct vimc_device {
>  	/* The Associated media_device parent */
>  	struct media_device mdev;
>  
> -	/* Internal v4l2 parent device*/
> +	/* Internal v4l2 parent device */
>  	struct v4l2_device v4l2_dev;
> +};
>  
> -	/* Subdevices */
> -	struct platform_device **subdevs;
> +/* Structure which describes individual configuration for each entity */
> +struct vimc_ent_config {
> +	const char *name;
> +	const char *drv;

I was tempted in suggesting to remove this field as it is not being used anymore,
but it will be usefull for the configfs patch.

Thanks
Helen

> +	struct vimc_ent_device *ved;
> +	int (*add)(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +	void (*rm)(struct vimc_device *vimc, struct vimc_ent_config *vent);
>  };
>  
> +/* prototypes for vimc_ent_config add and rm hooks */
> +int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +
> +int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +
> +int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +
> +int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent);
> +
>  #endif
> 

Re: [PATCH 2/3] media: vimc: Collapse component structure into a single monolithic driver

[Shuah Khan]

On 8/9/19 10:12 PM, Helen Koike wrote:
> Hi Shuah,
> 
> Thanks for the patch, just some small comments.
> 
> On 8/9/19 6:45 PM, Shuah Khan wrote:
>> vimc uses Component API to split the driver into functional components.
>> The real hardware resembles a monolith structure than component and
>> component structure added a level of complexity making it hard to
>> maintain without adding any real benefit.
>>
>> The sensor is one vimc component that would makes sense to be a separate
>> module to closely align with the real hardware. It would be easier to
>> collapse vimc into single monolithic driver first and then split the
>> sensor off as a separate module.
>>
>> Collapse it into a single monolithic driver removing the Component API.
>> This patch removes the component API and makes minimal changes to the
>> code base preserving the functional division of the code structure.
>> Preserving the functional structure allows us to split the sensor off
>> as a separate module in the future.
>>
>> Major design elements in this change are:
>> - Use existing struct vimc_ent_config and struct vimc_pipeline_config
>>    to drive the initialization of the functional components.
>> - Make vimc_ent_config global by moving it to vimc.h
>> - Add two new hooks add and rm to initialize and register, unregister
>>    and free subdevs.
>> - All component API is now gone and bind and unbind hooks are modified
>>    to do "add" and "rm" with minimal changes to just add and rm subdevs.
>> - vimc-core's bind and unbind are now register and unregister.
>> - vimc-core invokes "add" hooks from its vimc_register_devices().
>>    The "add" hooks remain the same and register subdevs. They don't
>>    create platform devices of their own and use vimc's pdev.dev as
>>    their reference device. The "add" hooks save their vimc_ent_device(s)
>>    in the corresponding vimc_ent_config.
>> - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
>>    and cleanup.
>> - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
>>    and the corresponding struct vimc_ent_config pointer.
>>
>> The following configure and stream test works on all devices.
>>
>> media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
>> media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
>> media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
>> media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
>>
>> v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
>> v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
>> v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
>>
>> v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>> v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>> v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>
>> Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
>> ---
>>   drivers/media/platform/vimc/Makefile       |   7 +-
>>   drivers/media/platform/vimc/vimc-capture.c |  75 ++--------
>>   drivers/media/platform/vimc/vimc-core.c    | 156 ++++++++-------------
>>   drivers/media/platform/vimc/vimc-debayer.c |  68 ++-------
>>   drivers/media/platform/vimc/vimc-scaler.c  |  68 ++-------
>>   drivers/media/platform/vimc/vimc-sensor.c  |  69 ++-------
>>   drivers/media/platform/vimc/vimc.h         |  25 +++-
>>   7 files changed, 125 insertions(+), 343 deletions(-)
>>
>> diff --git a/drivers/media/platform/vimc/Makefile b/drivers/media/platform/vimc/Makefile
>> index 96d06f030c31..a53b2b532e9f 100644
>> --- a/drivers/media/platform/vimc/Makefile
>> +++ b/drivers/media/platform/vimc/Makefile
>> @@ -1,5 +1,6 @@
>>   # SPDX-License-Identifier: GPL-2.0
>> -vimc-y := vimc-core.o vimc-common.o vimc-streamer.o
>> +vimc-y := vimc-core.o vimc-common.o vimc-streamer.o vimc-capture.o \
>> +		vimc-debayer.o vimc-scaler.o vimc-sensor.o
>> +
>> +obj-$(CONFIG_VIDEO_VIMC) += vimc.o
>>   
>> -obj-$(CONFIG_VIDEO_VIMC) += vimc.o vimc-capture.o vimc-debayer.o \
>> -                vimc-scaler.o vimc-sensor.o
>> diff --git a/drivers/media/platform/vimc/vimc-capture.c b/drivers/media/platform/vimc/vimc-capture.c
>> index c52fc5d97c2d..b7b2d3c3d4f8 100644
>> --- a/drivers/media/platform/vimc/vimc-capture.c
>> +++ b/drivers/media/platform/vimc/vimc-capture.c
>> @@ -5,10 +5,6 @@
>>    * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>>    */
>>   
>> -#include <linux/component.h>
>> -#include <linux/module.h>
>> -#include <linux/mod_devicetable.h>
>> -#include <linux/platform_device.h>
>>   #include <media/v4l2-ioctl.h>
>>   #include <media/videobuf2-core.h>
>>   #include <media/videobuf2-vmalloc.h>
>> @@ -17,8 +13,6 @@
>>   #include "vimc-common.h"
>>   #include "vimc-streamer.h"
>>   
>> -#define VIMC_CAP_DRV_NAME "vimc-capture"
>> -
>>   static const u32 vimc_cap_supported_pixfmt[] = {
>>   	V4L2_PIX_FMT_BGR24,
>>   	V4L2_PIX_FMT_RGB24,
>> @@ -348,11 +342,11 @@ static void vimc_cap_release(struct video_device *vdev)
>>   	kfree(vcap);
>>   }
>>   
>> -static void vimc_cap_comp_unbind(struct device *comp, struct device *master,
>> -				 void *master_data)
>> +void vimc_cap_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
>> -	struct vimc_cap_device *vcap = container_of(ved, struct vimc_cap_device,
>> +	struct vimc_ent_device *ved = vent->ved;
>> +	struct vimc_cap_device *vcap = container_of(ved,
>> +						    struct vimc_cap_device,
>>   						    ved);
>>   
>>   	vb2_queue_release(&vcap->queue);
>> @@ -399,11 +393,9 @@ static void *vimc_cap_process_frame(struct vimc_ent_device *ved,
>>   	return NULL;
>>   }
>>   
>> -static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>> -			      void *master_data)
>> +int vimc_cap_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct v4l2_device *v4l2_dev = master_data;
>> -	struct vimc_platform_data *pdata = comp->platform_data;
>> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>>   	struct vimc_cap_device *vcap;
>>   	struct video_device *vdev;
>>   	struct vb2_queue *q;
>> @@ -423,7 +415,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>>   	}
>>   
>>   	/* Initialize the media entity */
>> -	vcap->vdev.entity.name = pdata->entity_name;
>> +	vcap->vdev.entity.name = vent->name;
>>   	vcap->vdev.entity.function = MEDIA_ENT_F_IO_V4L;
>>   	ret = media_entity_pads_init(&vcap->vdev.entity,
>>   				     1, vcap->ved.pads);
>> @@ -447,8 +439,8 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>>   
>>   	ret = vb2_queue_init(q);
>>   	if (ret) {
>> -		dev_err(comp, "%s: vb2 queue init failed (err=%d)\n",
>> -			pdata->entity_name, ret);
>> +		dev_err(&vimc->pdev.dev, "%s: vb2 queue init failed (err=%d)\n",
>> +			vent->name, ret);
>>   		goto err_clean_m_ent;
>>   	}
>>   
>> @@ -465,8 +457,7 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>>   	vcap->ved.ent = &vcap->vdev.entity;
>>   	vcap->ved.process_frame = vimc_cap_process_frame;
>>   	vcap->ved.vdev_get_format = vimc_cap_get_format;
>> -	dev_set_drvdata(comp, &vcap->ved);
>> -	vcap->dev = comp;
>> +	vcap->dev = &vimc->pdev.dev;
>>   
>>   	/* Initialize the video_device struct */
>>   	vdev = &vcap->vdev;
>> @@ -479,17 +470,18 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>>   	vdev->queue = q;
>>   	vdev->v4l2_dev = v4l2_dev;
>>   	vdev->vfl_dir = VFL_DIR_RX;
>> -	strscpy(vdev->name, pdata->entity_name, sizeof(vdev->name));
>> +	strscpy(vdev->name, vent->name, sizeof(vdev->name));
>>   	video_set_drvdata(vdev, &vcap->ved);
>>   
>>   	/* Register the video_device with the v4l2 and the media framework */
>>   	ret = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
>>   	if (ret) {
>> -		dev_err(comp, "%s: video register failed (err=%d)\n",
>> +		dev_err(&vimc->pdev.dev, "%s: video register failed (err=%d)\n",
>>   			vcap->vdev.name, ret);
>>   		goto err_release_queue;
>>   	}
>>   
>> +	vent->ved = &vcap->ved;
>>   	return 0;
>>   
>>   err_release_queue:
>> @@ -503,44 +495,3 @@ static int vimc_cap_comp_bind(struct device *comp, struct device *master,
>>   
>>   	return ret;
>>   }
>> -
>> -static const struct component_ops vimc_cap_comp_ops = {
>> -	.bind = vimc_cap_comp_bind,
>> -	.unbind = vimc_cap_comp_unbind,
>> -};
>> -
>> -static int vimc_cap_probe(struct platform_device *pdev)
>> -{
>> -	return component_add(&pdev->dev, &vimc_cap_comp_ops);
>> -}
>> -
>> -static int vimc_cap_remove(struct platform_device *pdev)
>> -{
>> -	component_del(&pdev->dev, &vimc_cap_comp_ops);
>> -
>> -	return 0;
>> -}
>> -
>> -static const struct platform_device_id vimc_cap_driver_ids[] = {
>> -	{
>> -		.name           = VIMC_CAP_DRV_NAME,
>> -	},
>> -	{ }
>> -};
>> -
>> -static struct platform_driver vimc_cap_pdrv = {
>> -	.probe		= vimc_cap_probe,
>> -	.remove		= vimc_cap_remove,
>> -	.id_table	= vimc_cap_driver_ids,
>> -	.driver		= {
>> -		.name	= VIMC_CAP_DRV_NAME,
>> -	},
>> -};
>> -
>> -module_platform_driver(vimc_cap_pdrv);
>> -
>> -MODULE_DEVICE_TABLE(platform, vimc_cap_driver_ids);
>> -
>> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Capture");
>> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
>> -MODULE_LICENSE("GPL");
>> diff --git a/drivers/media/platform/vimc/vimc-core.c b/drivers/media/platform/vimc/vimc-core.c
>> index c9b351472272..3bd89cf38a90 100644
>> --- a/drivers/media/platform/vimc/vimc-core.c
>> +++ b/drivers/media/platform/vimc/vimc-core.c
>> @@ -5,7 +5,6 @@
>>    * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>>    */
>>   
>> -#include <linux/component.h>
>>   #include <linux/init.h>
>>   #include <linux/module.h>
>>   #include <linux/platform_device.h>
>> @@ -25,12 +24,6 @@
>>   	.flags = link_flags,					\
>>   }
>>   
>> -/* Structure which describes individual configuration for each entity */
>> -struct vimc_ent_config {
>> -	const char *name;
>> -	const char *drv;
>> -};
>> -
>>   /* Structure which describes links between entities */
>>   struct vimc_ent_link {
>>   	unsigned int src_ent;
>> @@ -42,7 +35,7 @@ struct vimc_ent_link {
>>   
>>   /* Structure which describes the whole topology */
>>   struct vimc_pipeline_config {
>> -	const struct vimc_ent_config *ents;
>> +	struct vimc_ent_config *ents;
>>   	size_t num_ents;
>>   	const struct vimc_ent_link *links;
>>   	size_t num_links;
>> @@ -52,43 +45,61 @@ struct vimc_pipeline_config {
>>    * Topology Configuration
>>    */
>>   
>> -static const struct vimc_ent_config ent_config[] = {
>> +static struct vimc_ent_config ent_config[] = {
>>   	{
>>   		.name = "Sensor A",
>>   		.drv = "vimc-sensor",
>> +		.add = vimc_sen_add,
>> +		.rm = vimc_sen_rm,
>>   	},
>>   	{
>>   		.name = "Sensor B",
>>   		.drv = "vimc-sensor",
>> +		.add = vimc_sen_add,
>> +		.rm = vimc_sen_rm,
>>   	},
>>   	{
>>   		.name = "Debayer A",
>>   		.drv = "vimc-debayer",
>> +		.add = vimc_deb_add,
>> +		.rm = vimc_deb_rm,
>>   	},
>>   	{
>>   		.name = "Debayer B",
>>   		.drv = "vimc-debayer",
>> +		.add = vimc_deb_add,
>> +		.rm = vimc_deb_rm,
>>   	},
>>   	{
>>   		.name = "Raw Capture 0",
>>   		.drv = "vimc-capture",
>> +		.add = vimc_cap_add,
>> +		.rm = vimc_cap_rm,
>>   	},
>>   	{
>>   		.name = "Raw Capture 1",
>>   		.drv = "vimc-capture",
>> +		.add = vimc_cap_add,
>> +		.rm = vimc_cap_rm,
>>   	},
>>   	{
>>   		.name = "RGB/YUV Input",
>>   		/* TODO: change this to vimc-input when it is implemented */
>>   		.drv = "vimc-sensor",
>> +		.add = vimc_sen_add,
>> +		.rm = vimc_sen_rm,
>>   	},
>>   	{
>>   		.name = "Scaler",
>>   		.drv = "vimc-scaler",
>> +		.add = vimc_sca_add,
>> +		.rm = vimc_sca_rm,
>>   	},
>>   	{
>>   		.name = "RGB/YUV Capture",
>>   		.drv = "vimc-capture",
>> +		.add = vimc_cap_add,
>> +		.rm = vimc_cap_rm,
>>   	},
>>   };
>>   
>> @@ -111,7 +122,7 @@ static const struct vimc_ent_link ent_links[] = {
>>   	VIMC_ENT_LINK(7, 1, 8, 0, MEDIA_LNK_FL_ENABLED | MEDIA_LNK_FL_IMMUTABLE),
>>   };
>>   
>> -static const struct vimc_pipeline_config pipe_cfg = {
>> +static struct vimc_pipeline_config pipe_cfg = {
> 
> Any special reason why the const was dropped here?
> 

Since this gets passed in .add and .rm hooks to be updated by subdevs
to save their struct vimc_ent_device for use by vimc-core, this needs
to writable. Without removing const, we will see warns.

>>   	.ents		= ent_config,
>>   	.num_ents	= ARRAY_SIZE(ent_config),
>>   	.links		= ent_links,
>> @@ -128,14 +139,11 @@ static int vimc_create_links(struct vimc_device *vimc)
>>   	/* Initialize the links between entities */
>>   	for (i = 0; i < vimc->pipe_cfg->num_links; i++) {
>>   		const struct vimc_ent_link *link = &vimc->pipe_cfg->links[i];
>> -		/*
>> -		 * TODO: Check another way of retrieving ved struct without
>> -		 * relying on platform_get_drvdata
>> -		 */
>> +
>>   		struct vimc_ent_device *ved_src =
>> -			platform_get_drvdata(vimc->subdevs[link->src_ent]);
>> +			vimc->pipe_cfg->ents[link->src_ent].ved;
>>   		struct vimc_ent_device *ved_sink =
>> -			platform_get_drvdata(vimc->subdevs[link->sink_ent]);
>> +			vimc->pipe_cfg->ents[link->sink_ent].ved;
>>   
>>   		ret = media_create_pad_link(ved_src->ent, link->src_pad,
>>   					    ved_sink->ent, link->sink_pad,
>> @@ -147,13 +155,28 @@ static int vimc_create_links(struct vimc_device *vimc)
>>   	return 0;
>>   }
>>   
>> -static int vimc_comp_bind(struct device *master)
>> +static int vimc_add_subdevs(struct vimc_device *vimc)
>>   {
>> -	struct vimc_device *vimc = container_of(to_platform_device(master),
>> -						struct vimc_device, pdev);
>> -	int ret;
>> +	int i;
> 
> unsigned int
> 
>> +	int ret = 0;
> 
> no need to init to 0, but not a problem either.
> 
>> +
>> +	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
>> +		dev_dbg(&vimc->pdev.dev, "new entity for %s\n",
>> +			vimc->pipe_cfg->ents[i].name);
>> +		ret = vimc->pipe_cfg->ents[i].add(vimc,
>> +				&vimc->pipe_cfg->ents[i]);
>> +		if (ret) {
>> +			dev_err(&vimc->pdev.dev, "add new entity for %s\n",
>> +				vimc->pipe_cfg->ents[i].name);
>> +			return ret;
>> +		}
>> +	}
>> +	return 0;
>> +}
>>   
>> -	dev_dbg(master, "bind");
>> +static int vimc_register_devices(struct vimc_device *vimc)
>> +{
>> +	int ret;
>>   
>>   	/* Register the v4l2 struct */
>>   	ret = v4l2_device_register(vimc->mdev.dev, &vimc->v4l2_dev);
>> @@ -163,22 +186,20 @@ static int vimc_comp_bind(struct device *master)
>>   		return ret;
>>   	}
>>   
>> -	/* Bind subdevices */
>> -	ret = component_bind_all(master, &vimc->v4l2_dev);
>> -	if (ret)
>> -		goto err_v4l2_unregister;
>> +	/* Invoke enitity config hooks to initialize and register subdevs */
>> +	vimc_add_subdevs(vimc);
>>   
>>   	/* Initialize links */
>>   	ret = vimc_create_links(vimc);
>>   	if (ret)
>> -		goto err_comp_unbind_all;
>> +		goto err_v4l2_unregister;
>>   
>>   	/* Register the media device */
>>   	ret = media_device_register(&vimc->mdev);
>>   	if (ret) {
>>   		dev_err(vimc->mdev.dev,
>>   			"media device register failed (err=%d)\n", ret);
>> -		goto err_comp_unbind_all;
>> +		goto err_v4l2_unregister;
>>   	}
>>   
>>   	/* Expose all subdev's nodes*/
>> @@ -195,98 +216,36 @@ static int vimc_comp_bind(struct device *master)
>>   err_mdev_unregister:
>>   	media_device_unregister(&vimc->mdev);
>>   	media_device_cleanup(&vimc->mdev);
>> -err_comp_unbind_all:
>> -	component_unbind_all(master, NULL);
>>   err_v4l2_unregister:
>>   	v4l2_device_unregister(&vimc->v4l2_dev);
>>   
>>   	return ret;
>>   }
>>   
>> -static void vimc_comp_unbind(struct device *master)
>> +static void vimc_unregister(struct vimc_device *vimc)
>>   {
>> -	struct vimc_device *vimc = container_of(to_platform_device(master),
>> -						struct vimc_device, pdev);
>> -
>> -	dev_dbg(master, "unbind");
>> -
>>   	media_device_unregister(&vimc->mdev);
>>   	media_device_cleanup(&vimc->mdev);
>> -	component_unbind_all(master, NULL);
>>   	v4l2_device_unregister(&vimc->v4l2_dev);
>>   }
>>   
>> -static int vimc_comp_compare(struct device *comp, void *data)
>> -{
>> -	return comp == data;
>> -}
>> -
>> -static struct component_match *vimc_add_subdevs(struct vimc_device *vimc)
>> -{
>> -	struct component_match *match = NULL;
>> -	struct vimc_platform_data pdata;
>> -	int i;
>> -
>> -	for (i = 0; i < vimc->pipe_cfg->num_ents; i++) {
>> -		dev_dbg(&vimc->pdev.dev, "new pdev for %s\n",
>> -			vimc->pipe_cfg->ents[i].drv);
>> -
>> -		strscpy(pdata.entity_name, vimc->pipe_cfg->ents[i].name,
>> -			sizeof(pdata.entity_name));
>> -
>> -		vimc->subdevs[i] = platform_device_register_data(&vimc->pdev.dev,
>> -						vimc->pipe_cfg->ents[i].drv,
>> -						PLATFORM_DEVID_AUTO,
>> -						&pdata,
>> -						sizeof(pdata));
>> -		if (IS_ERR(vimc->subdevs[i])) {
>> -			match = ERR_CAST(vimc->subdevs[i]);
>> -			while (--i >= 0)
>> -				platform_device_unregister(vimc->subdevs[i]);
>> -
>> -			return match;
>> -		}
>> -
>> -		component_match_add(&vimc->pdev.dev, &match, vimc_comp_compare,
>> -				    &vimc->subdevs[i]->dev);
>> -	}
>> -
>> -	return match;
>> -}
>> -
>>   static void vimc_rm_subdevs(struct vimc_device *vimc)
>>   {
>>   	unsigned int i;
>>   
>>   	for (i = 0; i < vimc->pipe_cfg->num_ents; i++)
>> -		platform_device_unregister(vimc->subdevs[i]);
>> +		vimc->pipe_cfg->ents[i].rm(vimc, &vimc->pipe_cfg->ents[i]);
>>   }
>>   
>> -static const struct component_master_ops vimc_comp_ops = {
>> -	.bind = vimc_comp_bind,
>> -	.unbind = vimc_comp_unbind,
>> -};
>> -
>>   static int vimc_probe(struct platform_device *pdev)
>>   {
>>   	struct vimc_device *vimc = container_of(pdev, struct vimc_device, pdev);
>> -	struct component_match *match = NULL;
>> -	int ret;
>> +	int ret = 0;
>>   
>>   	dev_dbg(&pdev->dev, "probe");
>>   
>>   	memset(&vimc->mdev, 0, sizeof(vimc->mdev));
>>   
>> -	/* Create platform_device for each entity in the topology*/
>> -	vimc->subdevs = devm_kcalloc(&vimc->pdev.dev, vimc->pipe_cfg->num_ents,
>> -				     sizeof(*vimc->subdevs), GFP_KERNEL);
>> -	if (!vimc->subdevs)
>> -		return -ENOMEM;
>> -
>> -	match = vimc_add_subdevs(vimc);
>> -	if (IS_ERR(match))
>> -		return PTR_ERR(match);
>> -
>>   	/* Link the media device within the v4l2_device */
>>   	vimc->v4l2_dev.mdev = &vimc->mdev;
>>   
>> @@ -298,16 +257,11 @@ static int vimc_probe(struct platform_device *pdev)
>>   	vimc->mdev.dev = &pdev->dev;
>>   	media_device_init(&vimc->mdev);
>>   
>> -	/* Add self to the component system */
>> -	ret = component_master_add_with_match(&pdev->dev, &vimc_comp_ops,
>> -					      match);
>> -	if (ret) {
>> +	ret = vimc_register_devices(vimc);
>> +	if (ret)
>>   		media_device_cleanup(&vimc->mdev);
>> -		vimc_rm_subdevs(vimc);
>> -		return ret;
>> -	}
>>   
>> -	return 0;
>> +	return ret;
>>   }
>>   
>>   static int vimc_remove(struct platform_device *pdev)
>> @@ -316,8 +270,8 @@ static int vimc_remove(struct platform_device *pdev)
>>   
>>   	dev_dbg(&pdev->dev, "remove");
>>   
>> -	component_master_del(&pdev->dev, &vimc_comp_ops);
>>   	vimc_rm_subdevs(vimc);
>> +	vimc_unregister(vimc);
>>   
>>   	return 0;
>>   }
>> diff --git a/drivers/media/platform/vimc/vimc-debayer.c b/drivers/media/platform/vimc/vimc-debayer.c
>> index 750752bb173c..b58e0946ac2a 100644
>> --- a/drivers/media/platform/vimc/vimc-debayer.c
>> +++ b/drivers/media/platform/vimc/vimc-debayer.c
>> @@ -5,10 +5,7 @@
>>    * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>>    */
>>   
>> -#include <linux/component.h>
>>   #include <linux/module.h>
>> -#include <linux/mod_devicetable.h>
>> -#include <linux/platform_device.h>
>>   #include <linux/vmalloc.h>
>>   #include <linux/v4l2-mediabus.h>
>>   #include <media/v4l2-subdev.h>
>> @@ -16,9 +13,7 @@
>>   #include "vimc.h"
>>   #include "vimc-common.h"
>>   
>> -#define VIMC_DEB_DRV_NAME "vimc-debayer"
>> -/* This module only supports transforming a bayer format
>> - * to V4L2_PIX_FMT_RGB24
>> +/* Supports transforming a bayer format to V4L2_PIX_FMT_RGB24
>>    */
>>   #define VIMC_DEB_SRC_PIXFMT V4L2_PIX_FMT_RGB24
>>   #define VIMC_DEB_SRC_MBUS_FMT_DEFAULT MEDIA_BUS_FMT_RGB888_1X24
>> @@ -499,21 +494,19 @@ static const struct v4l2_subdev_internal_ops vimc_deb_int_ops = {
>>   	.release = vimc_deb_release,
>>   };
>>   
>> -static void vimc_deb_comp_unbind(struct device *comp, struct device *master,
>> -				 void *master_data)
>> +void vimc_deb_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
>> -	struct vimc_deb_device *vdeb = container_of(ved, struct vimc_deb_device,
>> +	struct vimc_ent_device *ved = vent->ved;
>> +	struct vimc_deb_device *vdeb = container_of(ved,
>> +						    struct vimc_deb_device,
>>   						    ved);
>>   
>>   	vimc_ent_sd_unregister(ved, &vdeb->sd);
>>   }
>>   
>> -static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>> -			      void *master_data)
>> +int vimc_deb_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct v4l2_device *v4l2_dev = master_data;
>> -	struct vimc_platform_data *pdata = comp->platform_data;
>> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>>   	struct vimc_deb_device *vdeb;
>>   	int ret;
>>   
>> @@ -524,7 +517,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>>   
>>   	/* Initialize ved and sd */
>>   	ret = vimc_ent_sd_register(&vdeb->ved, &vdeb->sd, v4l2_dev,
>> -				   pdata->entity_name,
>> +				   vent->name,
>>   				   MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV, 2,
>>   				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
>>   				   MEDIA_PAD_FL_SOURCE},
>> @@ -535,8 +528,7 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>>   	}
>>   
>>   	vdeb->ved.process_frame = vimc_deb_process_frame;
>> -	dev_set_drvdata(comp, &vdeb->ved);
>> -	vdeb->dev = comp;
>> +	vdeb->dev = &vimc->pdev.dev;
>>   
>>   	/* Initialize the frame format */
>>   	vdeb->sink_fmt = sink_fmt_default;
>> @@ -549,46 +541,6 @@ static int vimc_deb_comp_bind(struct device *comp, struct device *master,
>>   	 */
>>   	vdeb->set_rgb_src = vimc_deb_set_rgb_pix_rgb24;
>>   
>> +	vent->ved = &vdeb->ved;
>>   	return 0;
>>   }
>> -
>> -static const struct component_ops vimc_deb_comp_ops = {
>> -	.bind = vimc_deb_comp_bind,
>> -	.unbind = vimc_deb_comp_unbind,
>> -};
>> -
>> -static int vimc_deb_probe(struct platform_device *pdev)
>> -{
>> -	return component_add(&pdev->dev, &vimc_deb_comp_ops);
>> -}
>> -
>> -static int vimc_deb_remove(struct platform_device *pdev)
>> -{
>> -	component_del(&pdev->dev, &vimc_deb_comp_ops);
>> -
>> -	return 0;
>> -}
>> -
>> -static const struct platform_device_id vimc_deb_driver_ids[] = {
>> -	{
>> -		.name           = VIMC_DEB_DRV_NAME,
>> -	},
>> -	{ }
>> -};
>> -
>> -static struct platform_driver vimc_deb_pdrv = {
>> -	.probe		= vimc_deb_probe,
>> -	.remove		= vimc_deb_remove,
>> -	.id_table	= vimc_deb_driver_ids,
>> -	.driver		= {
>> -		.name	= VIMC_DEB_DRV_NAME,
>> -	},
>> -};
>> -
>> -module_platform_driver(vimc_deb_pdrv);
>> -
>> -MODULE_DEVICE_TABLE(platform, vimc_deb_driver_ids);
>> -
>> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Debayer");
>> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
>> -MODULE_LICENSE("GPL");
>> diff --git a/drivers/media/platform/vimc/vimc-scaler.c b/drivers/media/platform/vimc/vimc-scaler.c
>> index fe99b9102ada..7cd478fee3ae 100644
>> --- a/drivers/media/platform/vimc/vimc-scaler.c
>> +++ b/drivers/media/platform/vimc/vimc-scaler.c
>> @@ -5,10 +5,7 @@
>>    * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>>    */
>>   
>> -#include <linux/component.h>
>>   #include <linux/module.h>
>> -#include <linux/mod_devicetable.h>
>> -#include <linux/platform_device.h>
>>   #include <linux/vmalloc.h>
>>   #include <linux/v4l2-mediabus.h>
>>   #include <media/v4l2-subdev.h>
>> @@ -16,8 +13,6 @@
>>   #include "vimc.h"
>>   #include "vimc-common.h"
>>   
>> -#define VIMC_SCA_DRV_NAME "vimc-scaler"
>> -
>>   static unsigned int sca_mult = 3;
>>   module_param(sca_mult, uint, 0000);
>>   MODULE_PARM_DESC(sca_mult, " the image size multiplier");
>> @@ -331,22 +326,21 @@ static const struct v4l2_subdev_internal_ops vimc_sca_int_ops = {
>>   	.release = vimc_sca_release,
>>   };
>>   
>> -static void vimc_sca_comp_unbind(struct device *comp, struct device *master,
>> -				 void *master_data)
>> +void vimc_sca_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
>> -	struct vimc_sca_device *vsca = container_of(ved, struct vimc_sca_device,
>> +	struct vimc_ent_device *ved = vent->ved;
>> +	struct vimc_sca_device *vsca = container_of(ved,
>> +						    struct vimc_sca_device,
>>   						    ved);
>>   
>> +
>>   	vimc_ent_sd_unregister(ved, &vsca->sd);
>>   }
>>   
>>   
>> -static int vimc_sca_comp_bind(struct device *comp, struct device *master,
>> -			      void *master_data)
>> +int vimc_sca_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct v4l2_device *v4l2_dev = master_data;
>> -	struct vimc_platform_data *pdata = comp->platform_data;
>> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>>   	struct vimc_sca_device *vsca;
>>   	int ret;
>>   
>> @@ -357,7 +351,7 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
>>   
>>   	/* Initialize ved and sd */
>>   	ret = vimc_ent_sd_register(&vsca->ved, &vsca->sd, v4l2_dev,
>> -				   pdata->entity_name,
>> +				   vent->name,
>>   				   MEDIA_ENT_F_PROC_VIDEO_SCALER, 2,
>>   				   (const unsigned long[2]) {MEDIA_PAD_FL_SINK,
>>   				   MEDIA_PAD_FL_SOURCE},
>> @@ -368,52 +362,12 @@ static int vimc_sca_comp_bind(struct device *comp, struct device *master,
>>   	}
>>   
>>   	vsca->ved.process_frame = vimc_sca_process_frame;
>> -	dev_set_drvdata(comp, &vsca->ved);
>> -	vsca->dev = comp;
>> +	vsca->dev = &vimc->pdev.dev;
>> +
>> +	vent->ved = &vsca->ved;
>>   
>>   	/* Initialize the frame format */
>>   	vsca->sink_fmt = sink_fmt_default;
>>   
>>   	return 0;
>>   }
>> -
>> -static const struct component_ops vimc_sca_comp_ops = {
>> -	.bind = vimc_sca_comp_bind,
>> -	.unbind = vimc_sca_comp_unbind,
>> -};
>> -
>> -static int vimc_sca_probe(struct platform_device *pdev)
>> -{
>> -	return component_add(&pdev->dev, &vimc_sca_comp_ops);
>> -}
>> -
>> -static int vimc_sca_remove(struct platform_device *pdev)
>> -{
>> -	component_del(&pdev->dev, &vimc_sca_comp_ops);
>> -
>> -	return 0;
>> -}
>> -
>> -static const struct platform_device_id vimc_sca_driver_ids[] = {
>> -	{
>> -		.name           = VIMC_SCA_DRV_NAME,
>> -	},
>> -	{ }
>> -};
>> -
>> -static struct platform_driver vimc_sca_pdrv = {
>> -	.probe		= vimc_sca_probe,
>> -	.remove		= vimc_sca_remove,
>> -	.id_table	= vimc_sca_driver_ids,
>> -	.driver		= {
>> -		.name	= VIMC_SCA_DRV_NAME,
>> -	},
>> -};
>> -
>> -module_platform_driver(vimc_sca_pdrv);
>> -
>> -MODULE_DEVICE_TABLE(platform, vimc_sca_driver_ids);
>> -
>> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Scaler");
>> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
>> -MODULE_LICENSE("GPL");
>> diff --git a/drivers/media/platform/vimc/vimc-sensor.c b/drivers/media/platform/vimc/vimc-sensor.c
>> index 6c57b1e262f9..bd237e030a1b 100644
>> --- a/drivers/media/platform/vimc/vimc-sensor.c
>> +++ b/drivers/media/platform/vimc/vimc-sensor.c
>> @@ -5,10 +5,6 @@
>>    * Copyright (C) 2015-2017 Helen Koike <helen.fornazier@gmail.com>
>>    */
>>   
>> -#include <linux/component.h>
>> -#include <linux/module.h>
>> -#include <linux/mod_devicetable.h>
>> -#include <linux/platform_device.h>
>>   #include <linux/v4l2-mediabus.h>
>>   #include <linux/vmalloc.h>
>>   #include <media/v4l2-ctrls.h>
>> @@ -19,8 +15,6 @@
>>   #include "vimc.h"
>>   #include "vimc-common.h"
>>   
>> -#define VIMC_SEN_DRV_NAME "vimc-sensor"
>> -
>>   static const struct v4l2_mbus_framefmt fmt_default = {
>>   	.width = 640,
>>   	.height = 480,
>> @@ -268,12 +262,12 @@ static const struct v4l2_subdev_internal_ops vimc_sen_int_ops = {
>>   	.release = vimc_sen_release,
>>   };
>>   
>> -static void vimc_sen_comp_unbind(struct device *comp, struct device *master,
>> -				 void *master_data)
>> +void vimc_sen_rm(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct vimc_ent_device *ved = dev_get_drvdata(comp);
>> -	struct vimc_sen_device *vsen =
>> -				container_of(ved, struct vimc_sen_device, ved);
>> +	struct vimc_ent_device *ved = vent->ved;
>> +	struct vimc_sen_device *vsen = container_of(ved,
>> +						    struct vimc_sen_device,
>> +						    ved);
>>   
>>   	vimc_ent_sd_unregister(ved, &vsen->sd);
>>   }
>> @@ -295,11 +289,9 @@ static const struct v4l2_ctrl_config vimc_sen_ctrl_test_pattern = {
>>   	.qmenu = tpg_pattern_strings,
>>   };
>>   
>> -static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>> -			      void *master_data)
>> +int vimc_sen_add(struct vimc_device *vimc, struct vimc_ent_config *vent)
>>   {
>> -	struct v4l2_device *v4l2_dev = master_data;
>> -	struct vimc_platform_data *pdata = comp->platform_data;
>> +	struct v4l2_device *v4l2_dev = &vimc->v4l2_dev;
>>   	struct vimc_sen_device *vsen;
>>   	int ret;
>>   
>> @@ -332,7 +324,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>>   
>>   	/* Initialize ved and sd */
>>   	ret = vimc_ent_sd_register(&vsen->ved, &vsen->sd, v4l2_dev,
>> -				   pdata->entity_name,
>> +				   vent->name,
>>   				   MEDIA_ENT_F_CAM_SENSOR, 1,
>>   				   (const unsigned long[1]) {MEDIA_PAD_FL_SOURCE},
>>   				   &vimc_sen_int_ops, &vimc_sen_ops);
>> @@ -340,8 +332,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>>   		goto err_free_hdl;
>>   
>>   	vsen->ved.process_frame = vimc_sen_process_frame;
>> -	dev_set_drvdata(comp, &vsen->ved);
>> -	vsen->dev = comp;
>> +	vsen->dev = &vimc->pdev.dev;
>>   
>>   	/* Initialize the frame format */
>>   	vsen->mbus_format = fmt_default;
>> @@ -353,6 +344,7 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>>   	if (ret)
>>   		goto err_unregister_ent_sd;
>>   
>> +	vent->ved = &vsen->ved;
>>   	return 0;
>>   
>>   err_unregister_ent_sd:
>> @@ -364,44 +356,3 @@ static int vimc_sen_comp_bind(struct device *comp, struct device *master,
>>   
>>   	return ret;
>>   }
>> -
>> -static const struct component_ops vimc_sen_comp_ops = {
>> -	.bind = vimc_sen_comp_bind,
>> -	.unbind = vimc_sen_comp_unbind,
>> -};
>> -
>> -static int vimc_sen_probe(struct platform_device *pdev)
>> -{
>> -	return component_add(&pdev->dev, &vimc_sen_comp_ops);
>> -}
>> -
>> -static int vimc_sen_remove(struct platform_device *pdev)
>> -{
>> -	component_del(&pdev->dev, &vimc_sen_comp_ops);
>> -
>> -	return 0;
>> -}
>> -
>> -static const struct platform_device_id vimc_sen_driver_ids[] = {
>> -	{
>> -		.name           = VIMC_SEN_DRV_NAME,
>> -	},
>> -	{ }
>> -};
>> -
>> -static struct platform_driver vimc_sen_pdrv = {
>> -	.probe		= vimc_sen_probe,
>> -	.remove		= vimc_sen_remove,
>> -	.id_table	= vimc_sen_driver_ids,
>> -	.driver		= {
>> -		.name	= VIMC_SEN_DRV_NAME,
>> -	},
>> -};
>> -
>> -module_platform_driver(vimc_sen_pdrv);
>> -
>> -MODULE_DEVICE_TABLE(platform, vimc_sen_driver_ids);
>> -
>> -MODULE_DESCRIPTION("Virtual Media Controller Driver (VIMC) Sensor");
>> -MODULE_AUTHOR("Helen Mae Koike Fornazier <helen.fornazier@gmail.com>");
>> -MODULE_LICENSE("GPL");
>> diff --git a/drivers/media/platform/vimc/vimc.h b/drivers/media/platform/vimc/vimc.h
>> index a5adebdda941..cd527c6fe6fc 100644
>> --- a/drivers/media/platform/vimc/vimc.h
>> +++ b/drivers/media/platform/vimc/vimc.h
>> @@ -92,11 +92,30 @@ struct vimc_device {
>>   	/* The Associated media_device parent */
>>   	struct media_device mdev;
>>   
>> -	/* Internal v4l2 parent device*/
>> +	/* Internal v4l2 parent device */
>>   	struct v4l2_device v4l2_dev;
>> +};
>>   
>> -	/* Subdevices */
>> -	struct platform_device **subdevs;
>> +/* Structure which describes individual configuration for each entity */
>> +struct vimc_ent_config {
>> +	const char *name;
>> +	const char *drv;
> 
> I was tempted in suggesting to remove this field as it is not being used anymore,
> but it will be usefull for the configfs patch.
> 

.drv isn't used anymore. I can remove that. name is still used.

thanks,
-- Shuah

[PATCH 3/3] media: vimc: Fix gpf in rmmod path when stream is active

[Shuah Khan]

If vimc module is removed while streaming is in progress, sensor
subdev unregister runs into general protection fault when it tries
to unregister media entities.

vimc_ent_sd_unregister() is fixed to call vimc_pads_cleanup() to
release media pads after v4l2_device_unregister_subdev() is done
unregistering the subdev and stop accessing media objects.

kernel: [ 4136.715839] general protection fault: 0000 [#1] SMP PTI
kernel: [ 4136.715847] CPU: 2 PID: 1972 Comm: bash Not tainted 5.3.0-rc2+ #4
kernel: [ 4136.715850] Hardware name: Dell Inc. OptiPlex 790/0HY9JP, BIOS A18 09/24/2013
kernel: [ 4136.715858] RIP: 0010:media_gobj_destroy.part.16+0x1f/0x60
kernel: [ 4136.715863] Code: ff 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 55 48 89 fe 48 89 e5 53 48 89 fb 48 c7 c7 00 7f cf b0 e8 24 fa ff ff 48 8b 03 <48> 83 80 a0 00 00 00 01 48 8b 43 18 48 8b 53 10 48 89 42 08 48 89
kernel: [ 4136.715866] RSP: 0018:ffff9b2248fe3cb0 EFLAGS: 00010246
kernel: [ 4136.715870] RAX: bcf2bfbfa0d63c2f RBX: ffff88c3eb37e9c0 RCX: 00000000802a0018
kernel: [ 4136.715873] RDX: ffff88c3e4f6a078 RSI: ffff88c3eb37e9c0 RDI: ffffffffb0cf7f00
kernel: [ 4136.715876] RBP: ffff9b2248fe3cb8 R08: 0000000001000002 R09: ffffffffb0492b00
kernel: [ 4136.715879] R10: ffff9b2248fe3c28 R11: 0000000000000001 R12: 0000000000000038
kernel: [ 4136.715881] R13: ffffffffc09a1628 R14: ffff88c3e4f6a028 R15: fffffffffffffff2
kernel: [ 4136.715885] FS:  00007f8389647740(0000) GS:ffff88c465500000(0000) knlGS:0000000000000000
kernel: [ 4136.715888] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kernel: [ 4136.715891] CR2: 000055d008f80fd8 CR3: 00000001996ec005 CR4: 00000000000606e0
kernel: [ 4136.715894] Call Trace:
kernel: [ 4136.715903]  media_gobj_destroy+0x14/0x20
kernel: [ 4136.715908]  __media_device_unregister_entity+0xb3/0xe0
kernel: [ 4136.715915]  media_device_unregister_entity+0x30/0x40
kernel: [ 4136.715920]  v4l2_device_unregister_subdev+0xa8/0xe0
kernel: [ 4136.715928]  vimc_ent_sd_unregister+0x1e/0x30 [vimc]
kernel: [ 4136.715933]  vimc_sen_rm+0x16/0x20 [vimc]
kernel: [ 4136.715938]  vimc_remove+0x3e/0xa0 [vimc]
kernel: [ 4136.715945]  platform_drv_remove+0x25/0x50
kernel: [ 4136.715951]  device_release_driver_internal+0xe0/0x1b0
kernel: [ 4136.715956]  device_driver_detach+0x14/0x20
kernel: [ 4136.715960]  unbind_store+0xd1/0x130
kernel: [ 4136.715965]  drv_attr_store+0x27/0x40
kernel: [ 4136.715971]  sysfs_kf_write+0x48/0x60
kernel: [ 4136.715976]  kernfs_fop_write+0x128/0x1b0
kernel: [ 4136.715982]  __vfs_write+0x1b/0x40
kernel: [ 4136.715987]  vfs_write+0xc3/0x1d0
kernel: [ 4136.715993]  ksys_write+0xaa/0xe0
kernel: [ 4136.715999]  __x64_sys_write+0x1a/0x20
kernel: [ 4136.716005]  do_syscall_64+0x5a/0x130
kernel: [ 4136.716010]  entry_SYSCALL_64_after_hwframe+0x4
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
---
 drivers/media/platform/vimc/vimc-common.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/media/platform/vimc/vimc-common.c b/drivers/media/platform/vimc/vimc-common.c
index 03016f204d05..abace5e0b4d3 100644
--- a/drivers/media/platform/vimc/vimc-common.c
+++ b/drivers/media/platform/vimc/vimc-common.c
@@ -373,7 +373,7 @@ EXPORT_SYMBOL_GPL(vimc_ent_sd_register);
 void vimc_ent_sd_unregister(struct vimc_ent_device *ved, struct v4l2_subdev *sd)
 {
 	media_entity_cleanup(ved->ent);
-	vimc_pads_cleanup(ved->pads);
 	v4l2_device_unregister_subdev(sd);
+	vimc_pads_cleanup(ved->pads);
 }
 EXPORT_SYMBOL_GPL(vimc_ent_sd_unregister);
-- 
2.17.1

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[André Almeida]

Hello Shuah,

Thanks for the patch, I did some comments below.

On 8/9/19 6:45 PM, Shuah Khan wrote:
> vimc uses Component API to split the driver into functional components.
> The real hardware resembles a monolith structure than component and
> component structure added a level of complexity making it hard to
> maintain without adding any real benefit.
>     
> The sensor is one vimc component that would makes sense to be a separate
> module to closely align with the real hardware. It would be easier to
> collapse vimc into single monolithic driver first and then split the
> sensor off as a separate module.
> 
> This patch series emoves the component API and makes minimal changes to
> the code base preserving the functional division of the code structure.
> Preserving the functional structure allows us to split the sensor off
> as a separate module in the future.
> 
> Major design elements in this change are:
>     - Use existing struct vimc_ent_config and struct vimc_pipeline_config
>       to drive the initialization of the functional components.
>     - Make vimc_ent_config global by moving it to vimc.h
>     - Add two new hooks add and rm to initialize and register, unregister
>       and free subdevs.
>     - All component API is now gone and bind and unbind hooks are modified
>       to do "add" and "rm" with minimal changes to just add and rm subdevs.
>     - vimc-core's bind and unbind are now register and unregister.
>     - vimc-core invokes "add" hooks from its vimc_register_devices().
>       The "add" hooks remain the same and register subdevs. They don't
>       create platform devices of their own and use vimc's pdev.dev as
>       their reference device. The "add" hooks save their vimc_ent_device(s)
>       in the corresponding vimc_ent_config.
>     - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
>       and cleanup.
>     - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
>       and the corresponding struct vimc_ent_config pointer.
>     
> The following configure and stream test works on all devices.
>     
>     media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
>     media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
>     media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
>     media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
>     
>     v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
>     v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
>     v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
>     
>     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>     v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
> 
> The third patch in the series fixes a general protection fault found
> when rmmod is done while stream is active.

I applied your patch on top of media_tree/master and I did some testing.
Not sure if I did something wrong, but just adding and removing the
module generated a kernel panic:

~# modprobe vimc
~# rmmod vimc
[   16.452974] stack segment: 0000 [#1] SMP PTI
[   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
[   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.12.0-20181126_142135-anatol 04/01/2014
[   16.456191] RIP: 0010:kfree+0x4d/0x240

<registers values...>

[   16.469188] Call Trace:
[   16.469666]  vimc_remove+0x35/0x90 [vimc]
[   16.470436]  platform_drv_remove+0x1f/0x40
[   16.471233]  device_release_driver_internal+0xd3/0x1b0
[   16.472184]  driver_detach+0x37/0x6b
[   16.472882]  bus_remove_driver+0x50/0xc1
[   16.473569]  vimc_exit+0xc/0xca0 [vimc]
[   16.474231]  __x64_sys_delete_module+0x18d/0x240
[   16.475036]  do_syscall_64+0x43/0x110
[   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[   16.476504] RIP: 0033:0x7fceb8dafa4b

<registers values...>

[   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
[   16.486187] ---[ end trace 91e5e0894e254d49 ]---
[   16.486758] RIP: 0010:kfree+0x4d/0x240

<registers values...>

fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)

I just added the module after booting, no other action was made. Here is
how my `git log --oneline` looks like:

897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
2e4a5ad8ad6d media: vimc: Collapse component structure into a single
monolithic driver
7c8da1687e92 media: vimc: move private defines to a common header
97299a303532 media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
...

> 
> vimc_print_dot (--print-dot) topology after this change:
> digraph board {
> 	rankdir=TB
> 	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000001:port0 -> n00000005:port0 [style=bold]
> 	n00000001:port0 -> n0000000b [style=bold]
> 	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000003:port0 -> n00000008:port0 [style=bold]
> 	n00000003:port0 -> n0000000f [style=bold]
> 	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000005:port1 -> n00000015:port0
> 	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000008:port1 -> n00000015:port0 [style=dashed]
> 	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
> 	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
> 	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000013:port0 -> n00000015:port0 [style=dashed]
> 	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
> 	n00000015:port1 -> n00000018 [style=bold]
> 	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
> }

Since the topology changed, it would be nice to change in the
documentation as well. The current dot file can be found at
`Documentation/media/v4l-drivers/vimc.dot` and it's rendered at this
page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html

Thanks,
	André

> 
> Shuah Khan (3):
>   media: vimc: move private defines to a common header
>   media: vimc: Collapse component structure into a single monolithic
>     driver
>   media: vimc: Fix gpf in rmmod path when stream is active
> 
>  drivers/media/platform/vimc/Makefile       |   7 +-
>  drivers/media/platform/vimc/vimc-capture.c |  96 ++----------
>  drivers/media/platform/vimc/vimc-common.c  |   2 +-
>  drivers/media/platform/vimc/vimc-core.c    | 174 +++++++--------------
>  drivers/media/platform/vimc/vimc-debayer.c |  84 ++--------
>  drivers/media/platform/vimc/vimc-scaler.c  |  83 ++--------
>  drivers/media/platform/vimc/vimc-sensor.c  |  82 ++--------
>  drivers/media/platform/vimc/vimc.h         | 121 ++++++++++++++
>  8 files changed, 230 insertions(+), 419 deletions(-)
>  create mode 100644 drivers/media/platform/vimc/vimc.h
> 

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

Hi Andre,

On 8/9/19 5:52 PM, André Almeida wrote:
> Hello Shuah,
> 
> Thanks for the patch, I did some comments below.
> 
> On 8/9/19 6:45 PM, Shuah Khan wrote:
>> vimc uses Component API to split the driver into functional components.
>> The real hardware resembles a monolith structure than component and
>> component structure added a level of complexity making it hard to
>> maintain without adding any real benefit.
>>      
>> The sensor is one vimc component that would makes sense to be a separate
>> module to closely align with the real hardware. It would be easier to
>> collapse vimc into single monolithic driver first and then split the
>> sensor off as a separate module.
>>
>> This patch series emoves the component API and makes minimal changes to
>> the code base preserving the functional division of the code structure.
>> Preserving the functional structure allows us to split the sensor off
>> as a separate module in the future.
>>
>> Major design elements in this change are:
>>      - Use existing struct vimc_ent_config and struct vimc_pipeline_config
>>        to drive the initialization of the functional components.
>>      - Make vimc_ent_config global by moving it to vimc.h
>>      - Add two new hooks add and rm to initialize and register, unregister
>>        and free subdevs.
>>      - All component API is now gone and bind and unbind hooks are modified
>>        to do "add" and "rm" with minimal changes to just add and rm subdevs.
>>      - vimc-core's bind and unbind are now register and unregister.
>>      - vimc-core invokes "add" hooks from its vimc_register_devices().
>>        The "add" hooks remain the same and register subdevs. They don't
>>        create platform devices of their own and use vimc's pdev.dev as
>>        their reference device. The "add" hooks save their vimc_ent_device(s)
>>        in the corresponding vimc_ent_config.
>>      - vimc-core invokes "rm" hooks from its unregister to unregister subdevs
>>        and cleanup.
>>      - vimc-core invokes "add" and "rm" hooks with pointer to struct vimc_device
>>        and the corresponding struct vimc_ent_config pointer.
>>      
>> The following configure and stream test works on all devices.
>>      
>>      media-ctl -d platform:vimc -V '"Sensor A":0[fmt:SBGGR8_1X8/640x480]'
>>      media-ctl -d platform:vimc -V '"Debayer A":0[fmt:SBGGR8_1X8/640x480]'
>>      media-ctl -d platform:vimc -V '"Sensor B":0[fmt:SBGGR8_1X8/640x480]'
>>      media-ctl -d platform:vimc -V '"Debayer B":0[fmt:SBGGR8_1X8/640x480]'
>>      
>>      v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v width=1920,height=1440
>>      v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
>>      v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
>>      
>>      v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>      v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>      v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>
>> The third patch in the series fixes a general protection fault found
>> when rmmod is done while stream is active.
> 
> I applied your patch on top of media_tree/master and I did some testing.
> Not sure if I did something wrong, but just adding and removing the
> module generated a kernel panic:

Thanks for testing.

Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
I will apply these to media latest and work from there. I have to
rebase these on top of the reverts from Lucas and Helen
> 
> ~# modprobe vimc
> ~# rmmod vimc
> [   16.452974] stack segment: 0000 [#1] SMP PTI
> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
> [   16.456191] RIP: 0010:kfree+0x4d/0x240
> 
> <registers values...>
> 
> [   16.469188] Call Trace:
> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
> [   16.470436]  platform_drv_remove+0x1f/0x40
> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
> [   16.472184]  driver_detach+0x37/0x6b
> [   16.472882]  bus_remove_driver+0x50/0xc1
> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
> [   16.475036]  do_syscall_64+0x43/0x110
> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [   16.476504] RIP: 0033:0x7fceb8dafa4b
> 
> <registers values...>
> 
> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
> [   16.486758] RIP: 0010:kfree+0x4d/0x240
> 
> <registers values...>
> 
> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
> 
> I just added the module after booting, no other action was made. Here is
> how my `git log --oneline` looks like:
> 
> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
> monolithic driver
> 7c8da1687e92 media: vimc: move private defines to a common header
> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
> ...
> 
>>
>> vimc_print_dot (--print-dot) topology after this change:
>> digraph board {
>> 	rankdir=TB
>> 	n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>> 	n00000001:port0 -> n00000005:port0 [style=bold]
>> 	n00000001:port0 -> n0000000b [style=bold]
>> 	n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>> 	n00000003:port0 -> n00000008:port0 [style=bold]
>> 	n00000003:port0 -> n0000000f [style=bold]
>> 	n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>> 	n00000005:port1 -> n00000015:port0
>> 	n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>> 	n00000008:port1 -> n00000015:port0 [style=dashed]
>> 	n0000000b [label="Raw Capture 0\n/dev/video1", shape=box, style=filled, fillcolor=yellow]
>> 	n0000000f [label="Raw Capture 1\n/dev/video2", shape=box, style=filled, fillcolor=yellow]
>> 	n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 | {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>> 	n00000013:port0 -> n00000015:port0 [style=dashed]
>> 	n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>> 	n00000015:port1 -> n00000018 [style=bold]
>> 	n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box, style=filled, fillcolor=yellow]
>> }
> 
> Since the topology changed, it would be nice to change in the
> documentation as well. The current dot file can be found at
> `Documentation/media/v4l-drivers/vimc.dot` and it's rendered at this
> page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html
> 

Topology shouldn't have changed. No changes to links or pads etc.
I will take a look to be sure. I agree that if topology changes
document should be updated.

thanks,
-- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[André Almeida]

On 8/9/19 9:17 PM, Shuah Khan wrote:
> Hi Andre,
> 
> On 8/9/19 5:52 PM, André Almeida wrote:
>> Hello Shuah,
>>
>> Thanks for the patch, I did some comments below.
>>
>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>> vimc uses Component API to split the driver into functional components.
>>> The real hardware resembles a monolith structure than component and
>>> component structure added a level of complexity making it hard to
>>> maintain without adding any real benefit.
>>>      The sensor is one vimc component that would makes sense to be a
>>> separate
>>> module to closely align with the real hardware. It would be easier to
>>> collapse vimc into single monolithic driver first and then split the
>>> sensor off as a separate module.
>>>
>>> This patch series emoves the component API and makes minimal changes to
>>> the code base preserving the functional division of the code structure.
>>> Preserving the functional structure allows us to split the sensor off
>>> as a separate module in the future.
>>>
>>> Major design elements in this change are:
>>>      - Use existing struct vimc_ent_config and struct
>>> vimc_pipeline_config
>>>        to drive the initialization of the functional components.
>>>      - Make vimc_ent_config global by moving it to vimc.h
>>>      - Add two new hooks add and rm to initialize and register,
>>> unregister
>>>        and free subdevs.
>>>      - All component API is now gone and bind and unbind hooks are
>>> modified
>>>        to do "add" and "rm" with minimal changes to just add and rm
>>> subdevs.
>>>      - vimc-core's bind and unbind are now register and unregister.
>>>      - vimc-core invokes "add" hooks from its vimc_register_devices().
>>>        The "add" hooks remain the same and register subdevs. They don't
>>>        create platform devices of their own and use vimc's pdev.dev as
>>>        their reference device. The "add" hooks save their
>>> vimc_ent_device(s)
>>>        in the corresponding vimc_ent_config.
>>>      - vimc-core invokes "rm" hooks from its unregister to unregister
>>> subdevs
>>>        and cleanup.
>>>      - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>> vimc_device
>>>        and the corresponding struct vimc_ent_config pointer.
>>>      The following configure and stream test works on all devices.
>>>           media-ctl -d platform:vimc -V '"Sensor
>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>      media-ctl -d platform:vimc -V '"Debayer
>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>      media-ctl -d platform:vimc -V '"Sensor
>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>      media-ctl -d platform:vimc -V '"Debayer
>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>           v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>> width=1920,height=1440
>>>      v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
>>>      v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
>>>           v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>      v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>      v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>
>>> The third patch in the series fixes a general protection fault found
>>> when rmmod is done while stream is active.
>>
>> I applied your patch on top of media_tree/master and I did some testing.
>> Not sure if I did something wrong, but just adding and removing the
>> module generated a kernel panic:
> 
> Thanks for testing.
> 
> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
> I will apply these to media latest and work from there. I have to
> rebase these on top of the reverts from Lucas and Helen

Ok, please let me know if I succeeded to reproduce.

>>
>> ~# modprobe vimc
>> ~# rmmod vimc
>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>
>> <registers values...>
>>
>> [   16.469188] Call Trace:
>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>> [   16.470436]  platform_drv_remove+0x1f/0x40
>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>> [   16.472184]  driver_detach+0x37/0x6b
>> [   16.472882]  bus_remove_driver+0x50/0xc1
>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>> [   16.475036]  do_syscall_64+0x43/0x110
>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>
>> <registers values...>
>>
>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>
>> <registers values...>
>>
>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
>>
>> I just added the module after booting, no other action was made. Here is
>> how my `git log --oneline` looks like:
>>
>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>> monolithic driver
>> 7c8da1687e92 media: vimc: move private defines to a common header
>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>> ...
>>
>>>
>>> vimc_print_dot (--print-dot) topology after this change:
>>> digraph board {
>>>     rankdir=TB
>>>     n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>     n00000001:port0 -> n00000005:port0 [style=bold]
>>>     n00000001:port0 -> n0000000b [style=bold]
>>>     n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>     n00000003:port0 -> n00000008:port0 [style=bold]
>>>     n00000003:port0 -> n0000000f [style=bold]
>>>     n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>     n00000005:port1 -> n00000015:port0
>>>     n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>     n00000008:port1 -> n00000015:port0 [style=dashed]
>>>     n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
>>> style=filled, fillcolor=yellow]
>>>     n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
>>> style=filled, fillcolor=yellow]
>>>     n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
>>> {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>     n00000013:port0 -> n00000015:port0 [style=dashed]
>>>     n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>     n00000015:port1 -> n00000018 [style=bold]
>>>     n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
>>> style=filled, fillcolor=yellow]
>>> }
>>
>> Since the topology changed, it would be nice to change in the
>> documentation as well. The current dot file can be found at
>> `Documentation/media/v4l-drivers/vimc.dot` and it's rendered at this
>> page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html
>>
> 
> Topology shouldn't have changed. No changes to links or pads etc.
> I will take a look to be sure. I agree that if topology changes
> document should be updated.

If you "diff" the current dot with the dot you generated, you will see
some differences. The main difference is that "RGB/YUV Input" was a
device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

> 
> thanks,
> -- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

On 8/9/19 6:24 PM, André Almeida wrote:
> On 8/9/19 9:17 PM, Shuah Khan wrote:
>> Hi Andre,
>>
>> On 8/9/19 5:52 PM, André Almeida wrote:
>>> Hello Shuah,
>>>
>>> Thanks for the patch, I did some comments below.
>>>
>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>> vimc uses Component API to split the driver into functional components.
>>>> The real hardware resembles a monolith structure than component and
>>>> component structure added a level of complexity making it hard to
>>>> maintain without adding any real benefit.
>>>>       The sensor is one vimc component that would makes sense to be a
>>>> separate
>>>> module to closely align with the real hardware. It would be easier to
>>>> collapse vimc into single monolithic driver first and then split the
>>>> sensor off as a separate module.
>>>>
>>>> This patch series emoves the component API and makes minimal changes to
>>>> the code base preserving the functional division of the code structure.
>>>> Preserving the functional structure allows us to split the sensor off
>>>> as a separate module in the future.
>>>>
>>>> Major design elements in this change are:
>>>>       - Use existing struct vimc_ent_config and struct
>>>> vimc_pipeline_config
>>>>         to drive the initialization of the functional components.
>>>>       - Make vimc_ent_config global by moving it to vimc.h
>>>>       - Add two new hooks add and rm to initialize and register,
>>>> unregister
>>>>         and free subdevs.
>>>>       - All component API is now gone and bind and unbind hooks are
>>>> modified
>>>>         to do "add" and "rm" with minimal changes to just add and rm
>>>> subdevs.
>>>>       - vimc-core's bind and unbind are now register and unregister.
>>>>       - vimc-core invokes "add" hooks from its vimc_register_devices().
>>>>         The "add" hooks remain the same and register subdevs. They don't
>>>>         create platform devices of their own and use vimc's pdev.dev as
>>>>         their reference device. The "add" hooks save their
>>>> vimc_ent_device(s)
>>>>         in the corresponding vimc_ent_config.
>>>>       - vimc-core invokes "rm" hooks from its unregister to unregister
>>>> subdevs
>>>>         and cleanup.
>>>>       - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>> vimc_device
>>>>         and the corresponding struct vimc_ent_config pointer.
>>>>       The following configure and stream test works on all devices.
>>>>            media-ctl -d platform:vimc -V '"Sensor
>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>       media-ctl -d platform:vimc -V '"Debayer
>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>       media-ctl -d platform:vimc -V '"Sensor
>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>       media-ctl -d platform:vimc -V '"Debayer
>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>> width=1920,height=1440
>>>>       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
>>>>       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
>>>>            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>
>>>> The third patch in the series fixes a general protection fault found
>>>> when rmmod is done while stream is active.
>>>
>>> I applied your patch on top of media_tree/master and I did some testing.
>>> Not sure if I did something wrong, but just adding and removing the
>>> module generated a kernel panic:
>>
>> Thanks for testing.
>>
>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>> I will apply these to media latest and work from there. I have to
>> rebase these on top of the reverts from Lucas and Helen
> 
> Ok, please let me know if I succeeded to reproduce.
> 
>>>
>>> ~# modprobe vimc
>>> ~# rmmod vimc
>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>
>>> <registers values...>
>>>
>>> [   16.469188] Call Trace:
>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>> [   16.472184]  driver_detach+0x37/0x6b
>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>> [   16.475036]  do_syscall_64+0x43/0x110
>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>
>>> <registers values...>
>>>
>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>
>>> <registers values...>
>>>
>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
>>>
>>> I just added the module after booting, no other action was made. Here is
>>> how my `git log --oneline` looks like:
>>>
>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>> monolithic driver
>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>> ...
>>>
>>>>
>>>> vimc_print_dot (--print-dot) topology after this change:
>>>> digraph board {
>>>>      rankdir=TB
>>>>      n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>      n00000001:port0 -> n00000005:port0 [style=bold]
>>>>      n00000001:port0 -> n0000000b [style=bold]
>>>>      n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>      n00000003:port0 -> n00000008:port0 [style=bold]
>>>>      n00000003:port0 -> n0000000f [style=bold]
>>>>      n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>      n00000005:port1 -> n00000015:port0
>>>>      n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>      n00000008:port1 -> n00000015:port0 [style=dashed]
>>>>      n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
>>>> style=filled, fillcolor=yellow]
>>>>      n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
>>>> style=filled, fillcolor=yellow]
>>>>      n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
>>>> {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>      n00000013:port0 -> n00000015:port0 [style=dashed]
>>>>      n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>      n00000015:port1 -> n00000018 [style=bold]
>>>>      n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
>>>> style=filled, fillcolor=yellow]
>>>> }
>>>
>>> Since the topology changed, it would be nice to change in the
>>> documentation as well. The current dot file can be found at
>>> `Documentation/media/v4l-drivers/vimc.dot` and it's rendered at this
>>> page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html
>>>
>>
>> Topology shouldn't have changed. No changes to links or pads etc.
>> I will take a look to be sure. I agree that if topology changes
>> document should be updated.
> 
> If you "diff" the current dot with the dot you generated, you will see
> some differences. The main difference is that "RGB/YUV Input" was a
> device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".
> 

Yeah. I should have saved dot before my changes and compare. The goal is
to not change anything. I will make sure topology doesn't change.

Thanks again for testing these patches quickly.

thanks,
-- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Helen Koike]

Hi Andre,

Thanks for testing this.

On 8/9/19 9:24 PM, André Almeida wrote:
> On 8/9/19 9:17 PM, Shuah Khan wrote:
>> Hi Andre,
>>
>> On 8/9/19 5:52 PM, André Almeida wrote:
>>> Hello Shuah,
>>>
>>> Thanks for the patch, I did some comments below.
>>>
>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>> vimc uses Component API to split the driver into functional components.
>>>> The real hardware resembles a monolith structure than component and
>>>> component structure added a level of complexity making it hard to
>>>> maintain without adding any real benefit.
>>>>      The sensor is one vimc component that would makes sense to be a
>>>> separate
>>>> module to closely align with the real hardware. It would be easier to
>>>> collapse vimc into single monolithic driver first and then split the
>>>> sensor off as a separate module.
>>>>
>>>> This patch series emoves the component API and makes minimal changes to
>>>> the code base preserving the functional division of the code structure.
>>>> Preserving the functional structure allows us to split the sensor off
>>>> as a separate module in the future.
>>>>
>>>> Major design elements in this change are:
>>>>      - Use existing struct vimc_ent_config and struct
>>>> vimc_pipeline_config
>>>>        to drive the initialization of the functional components.
>>>>      - Make vimc_ent_config global by moving it to vimc.h
>>>>      - Add two new hooks add and rm to initialize and register,
>>>> unregister
>>>>        and free subdevs.
>>>>      - All component API is now gone and bind and unbind hooks are
>>>> modified
>>>>        to do "add" and "rm" with minimal changes to just add and rm
>>>> subdevs.
>>>>      - vimc-core's bind and unbind are now register and unregister.
>>>>      - vimc-core invokes "add" hooks from its vimc_register_devices().
>>>>        The "add" hooks remain the same and register subdevs. They don't
>>>>        create platform devices of their own and use vimc's pdev.dev as
>>>>        their reference device. The "add" hooks save their
>>>> vimc_ent_device(s)
>>>>        in the corresponding vimc_ent_config.
>>>>      - vimc-core invokes "rm" hooks from its unregister to unregister
>>>> subdevs
>>>>        and cleanup.
>>>>      - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>> vimc_device
>>>>        and the corresponding struct vimc_ent_config pointer.
>>>>      The following configure and stream test works on all devices.
>>>>           media-ctl -d platform:vimc -V '"Sensor
>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>      media-ctl -d platform:vimc -V '"Debayer
>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>      media-ctl -d platform:vimc -V '"Sensor
>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>      media-ctl -d platform:vimc -V '"Debayer
>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>           v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>> width=1920,height=1440
>>>>      v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
>>>>      v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
>>>>           v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>      v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>      v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>
>>>> The third patch in the series fixes a general protection fault found
>>>> when rmmod is done while stream is active.
>>>
>>> I applied your patch on top of media_tree/master and I did some testing.
>>> Not sure if I did something wrong, but just adding and removing the
>>> module generated a kernel panic:
>>
>> Thanks for testing.
>>
>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>> I will apply these to media latest and work from there. I have to
>> rebase these on top of the reverts from Lucas and Helen
> 
> Ok, please let me know if I succeeded to reproduce.
> 
>>>
>>> ~# modprobe vimc
>>> ~# rmmod vimc
>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>
>>> <registers values...>
>>>
>>> [   16.469188] Call Trace:
>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>> [   16.472184]  driver_detach+0x37/0x6b
>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>> [   16.475036]  do_syscall_64+0x43/0x110
>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>
>>> <registers values...>
>>>
>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>
>>> <registers values...>
>>>
>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
>>>
>>> I just added the module after booting, no other action was made. Here is
>>> how my `git log --oneline` looks like:
>>>
>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>> monolithic driver
>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

I couldn't reproduce the error, my tree looks the same:

[I] koike@floko ~/m/o/linux> git log --oneline
e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
8a6d0b9adde0 media: vimc: move private defines to a common header
97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

André, is this deterministic? Or it just happens sometimes?

>>> ...
>>>
>>>>
>>>> vimc_print_dot (--print-dot) topology after this change:
>>>> digraph board {
>>>>     rankdir=TB
>>>>     n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>     n00000001:port0 -> n00000005:port0 [style=bold]
>>>>     n00000001:port0 -> n0000000b [style=bold]
>>>>     n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>     n00000003:port0 -> n00000008:port0 [style=bold]
>>>>     n00000003:port0 -> n0000000f [style=bold]
>>>>     n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>     n00000005:port1 -> n00000015:port0
>>>>     n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>     n00000008:port1 -> n00000015:port0 [style=dashed]
>>>>     n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
>>>> style=filled, fillcolor=yellow]
>>>>     n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
>>>> style=filled, fillcolor=yellow]
>>>>     n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
>>>> {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>     n00000013:port0 -> n00000015:port0 [style=dashed]
>>>>     n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>     n00000015:port1 -> n00000018 [style=bold]
>>>>     n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
>>>> style=filled, fillcolor=yellow]
>>>> }
>>>
>>> Since the topology changed, it would be nice to change in the
>>> documentation as well. The current dot file can be found at
>>> `Documentation/media/v4l-drivers/vimc.dot` and it's rendered at this
>>> page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html
>>>
>>
>> Topology shouldn't have changed. No changes to links or pads etc.
>> I will take a look to be sure. I agree that if topology changes
>> document should be updated.
> 
> If you "diff" the current dot with the dot you generated, you will see
> some differences. The main difference is that "RGB/YUV Input" was a
> device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".

hmm, I just generated the topology for media/master, and it is 
/dev/v4l-subdev4. As we don't have an implementation of the output device
yet, we used the sensor as a place holder and that is why it appears as
"/dev/v4l-subdev4", what is in the docs is the ideal version after we get
the output merged. We should update the docs in any case.

Regards,
Helen

> 
>>
>> thanks,
>> -- Shuah
> 

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

On 8/9/19 9:51 PM, Helen Koike wrote:
> Hi Andre,
> 
> Thanks for testing this.
> 
> On 8/9/19 9:24 PM, André Almeida wrote:
>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>> Hi Andre,
>>>
>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>> Hello Shuah,
>>>>
>>>> Thanks for the patch, I did some comments below.
>>>>
>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>> vimc uses Component API to split the driver into functional components.
>>>>> The real hardware resembles a monolith structure than component and
>>>>> component structure added a level of complexity making it hard to
>>>>> maintain without adding any real benefit.
>>>>>       The sensor is one vimc component that would makes sense to be a
>>>>> separate
>>>>> module to closely align with the real hardware. It would be easier to
>>>>> collapse vimc into single monolithic driver first and then split the
>>>>> sensor off as a separate module.
>>>>>
>>>>> This patch series emoves the component API and makes minimal changes to
>>>>> the code base preserving the functional division of the code structure.
>>>>> Preserving the functional structure allows us to split the sensor off
>>>>> as a separate module in the future.
>>>>>
>>>>> Major design elements in this change are:
>>>>>       - Use existing struct vimc_ent_config and struct
>>>>> vimc_pipeline_config
>>>>>         to drive the initialization of the functional components.
>>>>>       - Make vimc_ent_config global by moving it to vimc.h
>>>>>       - Add two new hooks add and rm to initialize and register,
>>>>> unregister
>>>>>         and free subdevs.
>>>>>       - All component API is now gone and bind and unbind hooks are
>>>>> modified
>>>>>         to do "add" and "rm" with minimal changes to just add and rm
>>>>> subdevs.
>>>>>       - vimc-core's bind and unbind are now register and unregister.
>>>>>       - vimc-core invokes "add" hooks from its vimc_register_devices().
>>>>>         The "add" hooks remain the same and register subdevs. They don't
>>>>>         create platform devices of their own and use vimc's pdev.dev as
>>>>>         their reference device. The "add" hooks save their
>>>>> vimc_ent_device(s)
>>>>>         in the corresponding vimc_ent_config.
>>>>>       - vimc-core invokes "rm" hooks from its unregister to unregister
>>>>> subdevs
>>>>>         and cleanup.
>>>>>       - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>>> vimc_device
>>>>>         and the corresponding struct vimc_ent_config pointer.
>>>>>       The following configure and stream test works on all devices.
>>>>>            media-ctl -d platform:vimc -V '"Sensor
>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>       media-ctl -d platform:vimc -V '"Debayer
>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>       media-ctl -d platform:vimc -V '"Sensor
>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>       media-ctl -d platform:vimc -V '"Debayer
>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>> width=1920,height=1440
>>>>>       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v pixelformat=BA81
>>>>>       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v pixelformat=BA81
>>>>>            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>>       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>
>>>>> The third patch in the series fixes a general protection fault found
>>>>> when rmmod is done while stream is active.
>>>>
>>>> I applied your patch on top of media_tree/master and I did some testing.
>>>> Not sure if I did something wrong, but just adding and removing the
>>>> module generated a kernel panic:
>>>
>>> Thanks for testing.
>>>
>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>> I will apply these to media latest and work from there. I have to
>>> rebase these on top of the reverts from Lucas and Helen
>>
>> Ok, please let me know if I succeeded to reproduce.
>>
>>>>
>>>> ~# modprobe vimc
>>>> ~# rmmod vimc
>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>
>>>> <registers values...>
>>>>
>>>> [   16.469188] Call Trace:
>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>
>>>> <registers values...>
>>>>
>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>
>>>> <registers values...>
>>>>
>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary error)
>>>>
>>>> I just added the module after booting, no other action was made. Here is
>>>> how my `git log --oneline` looks like:
>>>>
>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>> monolithic driver
>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
> 
> I couldn't reproduce the error, my tree looks the same:
> 
> [I] koike@floko ~/m/o/linux> git log --oneline
> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is active
> 43e9e2fe761f media: vimc: Collapse component structure into a single monolithic driver
> 8a6d0b9adde0 media: vimc: move private defines to a common header
> 97299a303532 (media/master) media: Remove dev_err() usage after platform_get_irq()
> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2

Thanks Helen for trying to reproduce and sharing the result.
> 
> André, is this deterministic? Or it just happens sometimes?
> 
>>>> ...
>>>>
>>>>>
>>>>> vimc_print_dot (--print-dot) topology after this change:
>>>>> digraph board {
>>>>>      rankdir=TB
>>>>>      n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
>>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>      n00000001:port0 -> n00000005:port0 [style=bold]
>>>>>      n00000001:port0 -> n0000000b [style=bold]
>>>>>      n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
>>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>      n00000003:port0 -> n00000008:port0 [style=bold]
>>>>>      n00000003:port0 -> n0000000f [style=bold]
>>>>>      n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
>>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>      n00000005:port1 -> n00000015:port0
>>>>>      n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
>>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>      n00000008:port1 -> n00000015:port0 [style=dashed]
>>>>>      n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
>>>>> style=filled, fillcolor=yellow]
>>>>>      n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
>>>>> style=filled, fillcolor=yellow]
>>>>>      n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
>>>>> {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>      n00000013:port0 -> n00000015:port0 [style=dashed]
>>>>>      n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
>>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>      n00000015:port1 -> n00000018 [style=bold]
>>>>>      n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
>>>>> style=filled, fillcolor=yellow]
>>>>> }
>>>>
>>>> Since the topology changed, it would be nice to change in the
>>>> documentation as well. The current dot file can be found at
>>>> `Documentation/media/v4l-drivers/vimc.dot` and it's rendered at this
>>>> page: https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html
>>>>
>>>

Thanks Andre! Yes this is the one I am using as a reference.

>>> Topology shouldn't have changed. No changes to links or pads etc.
>>> I will take a look to be sure. I agree that if topology changes
>>> document should be updated.
>>
>> If you "diff" the current dot with the dot you generated, you will see
>> some differences. The main difference is that "RGB/YUV Input" was a
>> device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".
> 
> hmm, I just generated the topology for media/master, and it is
> /dev/v4l-subdev4. As we don't have an implementation of the output device
> yet, we used the sensor as a place holder and that is why it appears as
> "/dev/v4l-subdev4", what is in the docs is the ideal version after we get
> the output merged. We should update the docs in any case.
> 

Helen! Does the master match what I generated. In any case, I will do
the diff of before and after my patches for sure.

I can update the document with what it should be.

thanks,
-- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[André Almeida]

Hi Shuah,

On 8/12/19 11:08 AM, Shuah Khan wrote:
> On 8/9/19 9:51 PM, Helen Koike wrote:
>> Hi Andre,
>>
>> Thanks for testing this.
>>
>> On 8/9/19 9:24 PM, André Almeida wrote:
>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>>> Hi Andre,
>>>>
>>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>>> Hello Shuah,
>>>>>
>>>>> Thanks for the patch, I did some comments below.
>>>>>
>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>>> vimc uses Component API to split the driver into functional
>>>>>> components.
>>>>>> The real hardware resembles a monolith structure than component and
>>>>>> component structure added a level of complexity making it hard to
>>>>>> maintain without adding any real benefit.
>>>>>>       The sensor is one vimc component that would makes sense to be a
>>>>>> separate
>>>>>> module to closely align with the real hardware. It would be easier to
>>>>>> collapse vimc into single monolithic driver first and then split the
>>>>>> sensor off as a separate module.
>>>>>>
>>>>>> This patch series emoves the component API and makes minimal
>>>>>> changes to
>>>>>> the code base preserving the functional division of the code
>>>>>> structure.
>>>>>> Preserving the functional structure allows us to split the sensor off
>>>>>> as a separate module in the future.
>>>>>>
>>>>>> Major design elements in this change are:
>>>>>>       - Use existing struct vimc_ent_config and struct
>>>>>> vimc_pipeline_config
>>>>>>         to drive the initialization of the functional components.
>>>>>>       - Make vimc_ent_config global by moving it to vimc.h
>>>>>>       - Add two new hooks add and rm to initialize and register,
>>>>>> unregister
>>>>>>         and free subdevs.
>>>>>>       - All component API is now gone and bind and unbind hooks are
>>>>>> modified
>>>>>>         to do "add" and "rm" with minimal changes to just add and rm
>>>>>> subdevs.
>>>>>>       - vimc-core's bind and unbind are now register and unregister.
>>>>>>       - vimc-core invokes "add" hooks from its
>>>>>> vimc_register_devices().
>>>>>>         The "add" hooks remain the same and register subdevs. They
>>>>>> don't
>>>>>>         create platform devices of their own and use vimc's
>>>>>> pdev.dev as
>>>>>>         their reference device. The "add" hooks save their
>>>>>> vimc_ent_device(s)
>>>>>>         in the corresponding vimc_ent_config.
>>>>>>       - vimc-core invokes "rm" hooks from its unregister to
>>>>>> unregister
>>>>>> subdevs
>>>>>>         and cleanup.
>>>>>>       - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>>>> vimc_device
>>>>>>         and the corresponding struct vimc_ent_config pointer.
>>>>>>       The following configure and stream test works on all devices.
>>>>>>            media-ctl -d platform:vimc -V '"Sensor
>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>       media-ctl -d platform:vimc -V '"Debayer
>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>       media-ctl -d platform:vimc -V '"Sensor
>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>       media-ctl -d platform:vimc -V '"Debayer
>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>            v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>>> width=1920,height=1440
>>>>>>       v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
>>>>>> pixelformat=BA81
>>>>>>       v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
>>>>>> pixelformat=BA81
>>>>>>            v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>>>       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>>       v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>>
>>>>>> The third patch in the series fixes a general protection fault found
>>>>>> when rmmod is done while stream is active.
>>>>>
>>>>> I applied your patch on top of media_tree/master and I did some
>>>>> testing.
>>>>> Not sure if I did something wrong, but just adding and removing the
>>>>> module generated a kernel panic:
>>>>
>>>> Thanks for testing.
>>>>
>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>>> I will apply these to media latest and work from there. I have to
>>>> rebase these on top of the reverts from Lucas and Helen
>>>
>>> Ok, please let me know if I succeeded to reproduce.
>>>
>>>>>
>>>>> ~# modprobe vimc
>>>>> ~# rmmod vimc
>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>>
>>>>> <registers values...>
>>>>>
>>>>> [   16.469188] Call Trace:
>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>>
>>>>> <registers values...>
>>>>>
>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>>
>>>>> <registers values...>
>>>>>
>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
>>>>> error)
>>>>>
>>>>> I just added the module after booting, no other action was made.
>>>>> Here is
>>>>> how my `git log --oneline` looks like:
>>>>>
>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>>> monolithic driver
>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>
>> I couldn't reproduce the error, my tree looks the same:
>>
>> [I] koike@floko ~/m/o/linux> git log --oneline
>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
>> active
>> 43e9e2fe761f media: vimc: Collapse component structure into a single
>> monolithic driver
>> 8a6d0b9adde0 media: vimc: move private defines to a common header
>> 97299a303532 (media/master) media: Remove dev_err() usage after
>> platform_get_irq()
>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
> 
> Thanks Helen for trying to reproduce and sharing the result.

Me and Helen found out what is the problem. If you follow this call trace:

vimc_ent_sd_unregister()
v4l2_device_unregister_subdev()
v4l2_subdev_release()

You'll notice that this last function calls the `release` callback
implementation of the subdevice. For instance, the `release` of
vimc-sensor is this one:

static void vimc_sen_release(struct v4l2_subdev *sd)
{
	struct vimc_sen_device *vsen =
				container_of(sd, struct vimc_sen_device, sd);

	v4l2_ctrl_handler_free(&vsen->hdl);
	tpg_free(&vsen->tpg);
	kfree(vsen);
}

And then you can see that `vsen` has been freed. Back to
vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
function will call vimc_pads_cleanup(). This is basically a
kfree(ved->pads), but `ved` has just been freed at
v4l2_subdev_release(), producing a memory fault.

To fix that, we found two options:

- place the kfree(ved->pads) inside the release callback of each
subdevice and removing vimc_pads_cleanup() from
vimc_ent_sd_unregister()
- use a auxiliary variable to hold the address of the pads, for instance:

void vimc_ent_sd_unregister(...)
{
    struct media_pad *pads = ved->pads;
    ...
    vimc_pads_cleanup(pads);
}


Thanks,
	André

>>
>> André, is this deterministic? Or it just happens sometimes?
>>
>>>>> ...
>>>>>
>>>>>>
>>>>>> vimc_print_dot (--print-dot) topology after this change:
>>>>>> digraph board {
>>>>>>      rankdir=TB
>>>>>>      n00000001 [label="{{} | Sensor A\n/dev/v4l-subdev0 | {<port0>
>>>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>>      n00000001:port0 -> n00000005:port0 [style=bold]
>>>>>>      n00000001:port0 -> n0000000b [style=bold]
>>>>>>      n00000003 [label="{{} | Sensor B\n/dev/v4l-subdev1 | {<port0>
>>>>>> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>>      n00000003:port0 -> n00000008:port0 [style=bold]
>>>>>>      n00000003:port0 -> n0000000f [style=bold]
>>>>>>      n00000005 [label="{{<port0> 0} | Debayer A\n/dev/v4l-subdev2 |
>>>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>>      n00000005:port1 -> n00000015:port0
>>>>>>      n00000008 [label="{{<port0> 0} | Debayer B\n/dev/v4l-subdev3 |
>>>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>>      n00000008:port1 -> n00000015:port0 [style=dashed]
>>>>>>      n0000000b [label="Raw Capture 0\n/dev/video1", shape=box,
>>>>>> style=filled, fillcolor=yellow]
>>>>>>      n0000000f [label="Raw Capture 1\n/dev/video2", shape=box,
>>>>>> style=filled, fillcolor=yellow]
>>>>>>      n00000013 [label="{{} | RGB/YUV Input\n/dev/v4l-subdev4 |
>>>>>> {<port0> 0}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>>      n00000013:port0 -> n00000015:port0 [style=dashed]
>>>>>>      n00000015 [label="{{<port0> 0} | Scaler\n/dev/v4l-subdev5 |
>>>>>> {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
>>>>>>      n00000015:port1 -> n00000018 [style=bold]
>>>>>>      n00000018 [label="RGB/YUV Capture\n/dev/video3", shape=box,
>>>>>> style=filled, fillcolor=yellow]
>>>>>> }
>>>>>
>>>>> Since the topology changed, it would be nice to change in the
>>>>> documentation as well. The current dot file can be found at
>>>>> `Documentation/media/v4l-drivers/vimc.dot` and it's rendered at this
>>>>> page:
>>>>> https://www.kernel.org/doc/html/latest/media/v4l-drivers/vimc.html
>>>>>
>>>>
> 
> Thanks Andre! Yes this is the one I am using as a reference.
> 
>>>> Topology shouldn't have changed. No changes to links or pads etc.
>>>> I will take a look to be sure. I agree that if topology changes
>>>> document should be updated.
>>>
>>> If you "diff" the current dot with the dot you generated, you will see
>>> some differences. The main difference is that "RGB/YUV Input" was a
>>> device "/dev/video2/", and now it a subdevice "/dev/v4l-subdev4".
>>
>> hmm, I just generated the topology for media/master, and it is
>> /dev/v4l-subdev4. As we don't have an implementation of the output device
>> yet, we used the sensor as a place holder and that is why it appears as
>> "/dev/v4l-subdev4", what is in the docs is the ideal version after we get
>> the output merged. We should update the docs in any case.
>>
> 
> Helen! Does the master match what I generated. In any case, I will do
> the diff of before and after my patches for sure.
> 
> I can update the document with what it should be.
> 
> thanks,
> -- Shuah
> 

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

On 8/12/19 12:52 PM, André Almeida wrote:
> Hi Shuah,
> 
> On 8/12/19 11:08 AM, Shuah Khan wrote:
>> On 8/9/19 9:51 PM, Helen Koike wrote:
>>> Hi Andre,
>>>
>>> Thanks for testing this.
>>>
>>> On 8/9/19 9:24 PM, André Almeida wrote:
>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>>>> Hi Andre,
>>>>>
>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>>>> Hello Shuah,
>>>>>>
>>>>>> Thanks for the patch, I did some comments below.
>>>>>>
>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>>>> vimc uses Component API to split the driver into functional
>>>>>>> components.
>>>>>>> The real hardware resembles a monolith structure than component and
>>>>>>> component structure added a level of complexity making it hard to
>>>>>>> maintain without adding any real benefit.
>>>>>>>        The sensor is one vimc component that would makes sense to be a
>>>>>>> separate
>>>>>>> module to closely align with the real hardware. It would be easier to
>>>>>>> collapse vimc into single monolithic driver first and then split the
>>>>>>> sensor off as a separate module.
>>>>>>>
>>>>>>> This patch series emoves the component API and makes minimal
>>>>>>> changes to
>>>>>>> the code base preserving the functional division of the code
>>>>>>> structure.
>>>>>>> Preserving the functional structure allows us to split the sensor off
>>>>>>> as a separate module in the future.
>>>>>>>
>>>>>>> Major design elements in this change are:
>>>>>>>        - Use existing struct vimc_ent_config and struct
>>>>>>> vimc_pipeline_config
>>>>>>>          to drive the initialization of the functional components.
>>>>>>>        - Make vimc_ent_config global by moving it to vimc.h
>>>>>>>        - Add two new hooks add and rm to initialize and register,
>>>>>>> unregister
>>>>>>>          and free subdevs.
>>>>>>>        - All component API is now gone and bind and unbind hooks are
>>>>>>> modified
>>>>>>>          to do "add" and "rm" with minimal changes to just add and rm
>>>>>>> subdevs.
>>>>>>>        - vimc-core's bind and unbind are now register and unregister.
>>>>>>>        - vimc-core invokes "add" hooks from its
>>>>>>> vimc_register_devices().
>>>>>>>          The "add" hooks remain the same and register subdevs. They
>>>>>>> don't
>>>>>>>          create platform devices of their own and use vimc's
>>>>>>> pdev.dev as
>>>>>>>          their reference device. The "add" hooks save their
>>>>>>> vimc_ent_device(s)
>>>>>>>          in the corresponding vimc_ent_config.
>>>>>>>        - vimc-core invokes "rm" hooks from its unregister to
>>>>>>> unregister
>>>>>>> subdevs
>>>>>>>          and cleanup.
>>>>>>>        - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>>>>> vimc_device
>>>>>>>          and the corresponding struct vimc_ent_config pointer.
>>>>>>>        The following configure and stream test works on all devices.
>>>>>>>             media-ctl -d platform:vimc -V '"Sensor
>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>        media-ctl -d platform:vimc -V '"Sensor
>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>             v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>>>> width=1920,height=1440
>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
>>>>>>> pixelformat=BA81
>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
>>>>>>> pixelformat=BA81
>>>>>>>             v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>>>
>>>>>>> The third patch in the series fixes a general protection fault found
>>>>>>> when rmmod is done while stream is active.
>>>>>>
>>>>>> I applied your patch on top of media_tree/master and I did some
>>>>>> testing.
>>>>>> Not sure if I did something wrong, but just adding and removing the
>>>>>> module generated a kernel panic:
>>>>>
>>>>> Thanks for testing.
>>>>>
>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>>>> I will apply these to media latest and work from there. I have to
>>>>> rebase these on top of the reverts from Lucas and Helen
>>>>
>>>> Ok, please let me know if I succeeded to reproduce.
>>>>
>>>>>>
>>>>>> ~# modprobe vimc
>>>>>> ~# rmmod vimc
>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>>>
>>>>>> <registers values...>
>>>>>>
>>>>>> [   16.469188] Call Trace:
>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>>>
>>>>>> <registers values...>
>>>>>>
>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>>>
>>>>>> <registers values...>
>>>>>>
>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
>>>>>> error)
>>>>>>
>>>>>> I just added the module after booting, no other action was made.
>>>>>> Here is
>>>>>> how my `git log --oneline` looks like:
>>>>>>
>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>>>> monolithic driver
>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>
>>> I couldn't reproduce the error, my tree looks the same:
>>>
>>> [I] koike@floko ~/m/o/linux> git log --oneline
>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
>>> active
>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
>>> monolithic driver
>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
>>> 97299a303532 (media/master) media: Remove dev_err() usage after
>>> platform_get_irq()
>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>
>> Thanks Helen for trying to reproduce and sharing the result.
> 
> Me and Helen found out what is the problem. If you follow this call trace:
> 
> vimc_ent_sd_unregister()
> v4l2_device_unregister_subdev()
> v4l2_subdev_release()
> 
> You'll notice that this last function calls the `release` callback
> implementation of the subdevice. For instance, the `release` of
> vimc-sensor is this one:
> 
> static void vimc_sen_release(struct v4l2_subdev *sd)
> {
> 	struct vimc_sen_device *vsen =
> 				container_of(sd, struct vimc_sen_device, sd);
> 
> 	v4l2_ctrl_handler_free(&vsen->hdl);
> 	tpg_free(&vsen->tpg);
> 	kfree(vsen);
> }
> 
> And then you can see that `vsen` has been freed. Back to
> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
> function will call vimc_pads_cleanup(). This is basically a
> kfree(ved->pads), but `ved` has just been freed at
> v4l2_subdev_release(), producing a memory fault.
> 
> To fix that, we found two options:
> 
> - place the kfree(ved->pads) inside the release callback of each
> subdevice and removing vimc_pads_cleanup() from
> vimc_ent_sd_unregister()
> - use a auxiliary variable to hold the address of the pads, for instance:
> 
> void vimc_ent_sd_unregister(...)
> {
>      struct media_pad *pads = ved->pads;
>      ...
>      vimc_pads_cleanup(pads);
> }
> 
> 

I fixed a problem in the thirds patch. vimc-capture uses the first
approach - placing the kfree(ved->pads) inside the release callback.

I am debugging another such problem in unbind path while streaming.
I am working on v2 and I will look for the rmmod problem and fix it.

thanks again for testing and finding the root cause.
-- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

On 8/12/19 1:10 PM, Shuah Khan wrote:
> On 8/12/19 12:52 PM, André Almeida wrote:
>> Hi Shuah,
>>
>> On 8/12/19 11:08 AM, Shuah Khan wrote:
>>> On 8/9/19 9:51 PM, Helen Koike wrote:
>>>> Hi Andre,
>>>>
>>>> Thanks for testing this.
>>>>
>>>> On 8/9/19 9:24 PM, André Almeida wrote:
>>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>>>>> Hi Andre,
>>>>>>
>>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>>>>> Hello Shuah,
>>>>>>>
>>>>>>> Thanks for the patch, I did some comments below.
>>>>>>>
>>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>>>>> vimc uses Component API to split the driver into functional
>>>>>>>> components.
>>>>>>>> The real hardware resembles a monolith structure than component and
>>>>>>>> component structure added a level of complexity making it hard to
>>>>>>>> maintain without adding any real benefit.
>>>>>>>>        The sensor is one vimc component that would makes sense 
>>>>>>>> to be a
>>>>>>>> separate
>>>>>>>> module to closely align with the real hardware. It would be 
>>>>>>>> easier to
>>>>>>>> collapse vimc into single monolithic driver first and then split 
>>>>>>>> the
>>>>>>>> sensor off as a separate module.
>>>>>>>>
>>>>>>>> This patch series emoves the component API and makes minimal
>>>>>>>> changes to
>>>>>>>> the code base preserving the functional division of the code
>>>>>>>> structure.
>>>>>>>> Preserving the functional structure allows us to split the 
>>>>>>>> sensor off
>>>>>>>> as a separate module in the future.
>>>>>>>>
>>>>>>>> Major design elements in this change are:
>>>>>>>>        - Use existing struct vimc_ent_config and struct
>>>>>>>> vimc_pipeline_config
>>>>>>>>          to drive the initialization of the functional components.
>>>>>>>>        - Make vimc_ent_config global by moving it to vimc.h
>>>>>>>>        - Add two new hooks add and rm to initialize and register,
>>>>>>>> unregister
>>>>>>>>          and free subdevs.
>>>>>>>>        - All component API is now gone and bind and unbind hooks 
>>>>>>>> are
>>>>>>>> modified
>>>>>>>>          to do "add" and "rm" with minimal changes to just add 
>>>>>>>> and rm
>>>>>>>> subdevs.
>>>>>>>>        - vimc-core's bind and unbind are now register and 
>>>>>>>> unregister.
>>>>>>>>        - vimc-core invokes "add" hooks from its
>>>>>>>> vimc_register_devices().
>>>>>>>>          The "add" hooks remain the same and register subdevs. They
>>>>>>>> don't
>>>>>>>>          create platform devices of their own and use vimc's
>>>>>>>> pdev.dev as
>>>>>>>>          their reference device. The "add" hooks save their
>>>>>>>> vimc_ent_device(s)
>>>>>>>>          in the corresponding vimc_ent_config.
>>>>>>>>        - vimc-core invokes "rm" hooks from its unregister to
>>>>>>>> unregister
>>>>>>>> subdevs
>>>>>>>>          and cleanup.
>>>>>>>>        - vimc-core invokes "add" and "rm" hooks with pointer to 
>>>>>>>> struct
>>>>>>>> vimc_device
>>>>>>>>          and the corresponding struct vimc_ent_config pointer.
>>>>>>>>        The following configure and stream test works on all 
>>>>>>>> devices.
>>>>>>>>             media-ctl -d platform:vimc -V '"Sensor
>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>        media-ctl -d platform:vimc -V '"Sensor
>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>             v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>>>>> width=1920,height=1440
>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
>>>>>>>> pixelformat=BA81
>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
>>>>>>>> pixelformat=BA81
>>>>>>>>             v4l2-ctl --stream-mmap --stream-count=100 -d 
>>>>>>>> /dev/video1
>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>>>>
>>>>>>>> The third patch in the series fixes a general protection fault 
>>>>>>>> found
>>>>>>>> when rmmod is done while stream is active.
>>>>>>>
>>>>>>> I applied your patch on top of media_tree/master and I did some
>>>>>>> testing.
>>>>>>> Not sure if I did something wrong, but just adding and removing the
>>>>>>> module generated a kernel panic:
>>>>>>
>>>>>> Thanks for testing.
>>>>>>
>>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>>>>> I will apply these to media latest and work from there. I have to
>>>>>> rebase these on top of the reverts from Lucas and Helen
>>>>>
>>>>> Ok, please let me know if I succeeded to reproduce.
>>>>>
>>>>>>>
>>>>>>> ~# modprobe vimc
>>>>>>> ~# rmmod vimc
>>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 
>>>>>>> 5.3.0-rc2+ #36
>>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 
>>>>>>> 1996),
>>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>>>>
>>>>>>> <registers values...>
>>>>>>>
>>>>>>> [   16.469188] Call Trace:
>>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>>>>
>>>>>>> <registers values...>
>>>>>>>
>>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>>>>
>>>>>>> <registers values...>
>>>>>>>
>>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
>>>>>>> error)
>>>>>>>
>>>>>>> I just added the module after booting, no other action was made.
>>>>>>> Here is
>>>>>>> how my `git log --oneline` looks like:
>>>>>>>
>>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is 
>>>>>>> active
>>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>>>>> monolithic driver
>>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to 
>>>>>>> adv7511-v4l2
>>>>
>>>> I couldn't reproduce the error, my tree looks the same:
>>>>
>>>> [I] koike@floko ~/m/o/linux> git log --oneline
>>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
>>>> active
>>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
>>>> monolithic driver
>>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
>>>> 97299a303532 (media/master) media: Remove dev_err() usage after
>>>> platform_get_irq()
>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>
>>> Thanks Helen for trying to reproduce and sharing the result.
>>
>> Me and Helen found out what is the problem. If you follow this call 
>> trace:
>>
>> vimc_ent_sd_unregister()
>> v4l2_device_unregister_subdev()
>> v4l2_subdev_release()
>>
>> You'll notice that this last function calls the `release` callback
>> implementation of the subdevice. For instance, the `release` of
>> vimc-sensor is this one:
>>
>> static void vimc_sen_release(struct v4l2_subdev *sd)
>> {
>>     struct vimc_sen_device *vsen =
>>                 container_of(sd, struct vimc_sen_device, sd);
>>
>>     v4l2_ctrl_handler_free(&vsen->hdl);
>>     tpg_free(&vsen->tpg);
>>     kfree(vsen);
>> }
>>
>> And then you can see that `vsen` has been freed. Back to
>> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
>> function will call vimc_pads_cleanup(). This is basically a
>> kfree(ved->pads), but `ved` has just been freed at
>> v4l2_subdev_release(), producing a memory fault.
>>
>> To fix that, we found two options:
>>
>> - place the kfree(ved->pads) inside the release callback of each
>> subdevice and removing vimc_pads_cleanup() from
>> vimc_ent_sd_unregister()
>> - use a auxiliary variable to hold the address of the pads, for instance:
>>
>> void vimc_ent_sd_unregister(...)
>> {
>>      struct media_pad *pads = ved->pads;
>>      ...
>>      vimc_pads_cleanup(pads);
>> }
>>
>>
> 
> I fixed a problem in the thirds patch. vimc-capture uses the first
> approach - placing the kfree(ved->pads) inside the release callback.
> 
> I am debugging another such problem in unbind path while streaming.
> I am working on v2 and I will look for the rmmod problem and fix it.
> 
> thanks again for testing and finding the root cause.
> -- Shuah

Hi Andre,

Here is what's happening.

Before this change, you can't really do rmmod vimc, because vimc is in
use by other component drivers. With the collapse, now you can actually
do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees
pads first and the does v4l2_device_unregister_subdev().

I fixed this in the 3/3 patch. I can reproduce the problem with patches 
1 and 2, and patch 3 fixes it.

Did you test with the third patch in this series?

thanks,
-- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Helen Koike]

Hi Shuah,

On 8/12/19 7:14 PM, Shuah Khan wrote:
> On 8/12/19 1:10 PM, Shuah Khan wrote:
>> On 8/12/19 12:52 PM, André Almeida wrote:
>>> Hi Shuah,
>>>
>>> On 8/12/19 11:08 AM, Shuah Khan wrote:
>>>> On 8/9/19 9:51 PM, Helen Koike wrote:
>>>>> Hi Andre,
>>>>>
>>>>> Thanks for testing this.
>>>>>
>>>>> On 8/9/19 9:24 PM, André Almeida wrote:
>>>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>>>>>> Hi Andre,
>>>>>>>
>>>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>>>>>> Hello Shuah,
>>>>>>>>
>>>>>>>> Thanks for the patch, I did some comments below.
>>>>>>>>
>>>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>>>>>> vimc uses Component API to split the driver into functional
>>>>>>>>> components.
>>>>>>>>> The real hardware resembles a monolith structure than component and
>>>>>>>>> component structure added a level of complexity making it hard to
>>>>>>>>> maintain without adding any real benefit.
>>>>>>>>>        The sensor is one vimc component that would makes sense to be a
>>>>>>>>> separate
>>>>>>>>> module to closely align with the real hardware. It would be easier to
>>>>>>>>> collapse vimc into single monolithic driver first and then split the
>>>>>>>>> sensor off as a separate module.
>>>>>>>>>
>>>>>>>>> This patch series emoves the component API and makes minimal
>>>>>>>>> changes to
>>>>>>>>> the code base preserving the functional division of the code
>>>>>>>>> structure.
>>>>>>>>> Preserving the functional structure allows us to split the sensor off
>>>>>>>>> as a separate module in the future.
>>>>>>>>>
>>>>>>>>> Major design elements in this change are:
>>>>>>>>>        - Use existing struct vimc_ent_config and struct
>>>>>>>>> vimc_pipeline_config
>>>>>>>>>          to drive the initialization of the functional components.
>>>>>>>>>        - Make vimc_ent_config global by moving it to vimc.h
>>>>>>>>>        - Add two new hooks add and rm to initialize and register,
>>>>>>>>> unregister
>>>>>>>>>          and free subdevs.
>>>>>>>>>        - All component API is now gone and bind and unbind hooks are
>>>>>>>>> modified
>>>>>>>>>          to do "add" and "rm" with minimal changes to just add and rm
>>>>>>>>> subdevs.
>>>>>>>>>        - vimc-core's bind and unbind are now register and unregister.
>>>>>>>>>        - vimc-core invokes "add" hooks from its
>>>>>>>>> vimc_register_devices().
>>>>>>>>>          The "add" hooks remain the same and register subdevs. They
>>>>>>>>> don't
>>>>>>>>>          create platform devices of their own and use vimc's
>>>>>>>>> pdev.dev as
>>>>>>>>>          their reference device. The "add" hooks save their
>>>>>>>>> vimc_ent_device(s)
>>>>>>>>>          in the corresponding vimc_ent_config.
>>>>>>>>>        - vimc-core invokes "rm" hooks from its unregister to
>>>>>>>>> unregister
>>>>>>>>> subdevs
>>>>>>>>>          and cleanup.
>>>>>>>>>        - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>>>>>>> vimc_device
>>>>>>>>>          and the corresponding struct vimc_ent_config pointer.
>>>>>>>>>        The following configure and stream test works on all devices.
>>>>>>>>>             media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>        media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>             v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>>>>>> width=1920,height=1440
>>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
>>>>>>>>> pixelformat=BA81
>>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
>>>>>>>>> pixelformat=BA81
>>>>>>>>>             v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>>>>>
>>>>>>>>> The third patch in the series fixes a general protection fault found
>>>>>>>>> when rmmod is done while stream is active.
>>>>>>>>
>>>>>>>> I applied your patch on top of media_tree/master and I did some
>>>>>>>> testing.
>>>>>>>> Not sure if I did something wrong, but just adding and removing the
>>>>>>>> module generated a kernel panic:
>>>>>>>
>>>>>>> Thanks for testing.
>>>>>>>
>>>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>>>>>> I will apply these to media latest and work from there. I have to
>>>>>>> rebase these on top of the reverts from Lucas and Helen
>>>>>>
>>>>>> Ok, please let me know if I succeeded to reproduce.
>>>>>>
>>>>>>>>
>>>>>>>> ~# modprobe vimc
>>>>>>>> ~# rmmod vimc
>>>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>
>>>>>>>> <registers values...>
>>>>>>>>
>>>>>>>> [   16.469188] Call Trace:
>>>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>>>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>>>>>
>>>>>>>> <registers values...>
>>>>>>>>
>>>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>
>>>>>>>> <registers values...>
>>>>>>>>
>>>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
>>>>>>>> error)
>>>>>>>>
>>>>>>>> I just added the module after booting, no other action was made.
>>>>>>>> Here is
>>>>>>>> how my `git log --oneline` looks like:
>>>>>>>>
>>>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>>>>>> monolithic driver
>>>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>>
>>>>> I couldn't reproduce the error, my tree looks the same:
>>>>>
>>>>> [I] koike@floko ~/m/o/linux> git log --oneline
>>>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
>>>>> active
>>>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
>>>>> monolithic driver
>>>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
>>>>> 97299a303532 (media/master) media: Remove dev_err() usage after
>>>>> platform_get_irq()
>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>
>>>> Thanks Helen for trying to reproduce and sharing the result.
>>>
>>> Me and Helen found out what is the problem. If you follow this call trace:
>>>
>>> vimc_ent_sd_unregister()
>>> v4l2_device_unregister_subdev()
>>> v4l2_subdev_release()
>>>
>>> You'll notice that this last function calls the `release` callback
>>> implementation of the subdevice. For instance, the `release` of
>>> vimc-sensor is this one:
>>>
>>> static void vimc_sen_release(struct v4l2_subdev *sd)
>>> {
>>>     struct vimc_sen_device *vsen =
>>>                 container_of(sd, struct vimc_sen_device, sd);
>>>
>>>     v4l2_ctrl_handler_free(&vsen->hdl);
>>>     tpg_free(&vsen->tpg);
>>>     kfree(vsen);
>>> }
>>>
>>> And then you can see that `vsen` has been freed. Back to
>>> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
>>> function will call vimc_pads_cleanup(). This is basically a
>>> kfree(ved->pads), but `ved` has just been freed at
>>> v4l2_subdev_release(), producing a memory fault.
>>>
>>> To fix that, we found two options:
>>>
>>> - place the kfree(ved->pads) inside the release callback of each
>>> subdevice and removing vimc_pads_cleanup() from
>>> vimc_ent_sd_unregister()
>>> - use a auxiliary variable to hold the address of the pads, for instance:
>>>
>>> void vimc_ent_sd_unregister(...)
>>> {
>>>      struct media_pad *pads = ved->pads;
>>>      ...
>>>      vimc_pads_cleanup(pads);
>>> }
>>>
>>>
>>
>> I fixed a problem in the thirds patch. vimc-capture uses the first
>> approach - placing the kfree(ved->pads) inside the release callback.
>>
>> I am debugging another such problem in unbind path while streaming.
>> I am working on v2 and I will look for the rmmod problem and fix it.
>>
>> thanks again for testing and finding the root cause.
>> -- Shuah
> 
> Hi Andre,
> 
> Here is what's happening.
> 
> Before this change, you can't really do rmmod vimc, because vimc is in
> use by other component drivers. With the collapse, now you can actually
> do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees
> pads first and the does v4l2_device_unregister_subdev().
> 
> I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
> 
> Did you test with the third patch in this series?

yes, we tested with 3/3, but the new problem now is when doing the following
in this order:

    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(ved->pads);


v4l2_device_unregister_subdev() calls the release function of the subdevice that
frees the ved object, so ved->pads is not valid anymore. That is why André suggested
a temporary variable to hold ved->pads and to be able to free it later:

    struct media_pad *pads = ved->pads;

    v4l2_device_unregister_subdev(sd);
    vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.


Regards,
Helen

> 
> thanks,
> -- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

On 8/12/19 5:41 PM, Helen Koike wrote:
> Hi Shuah,
> 
> On 8/12/19 7:14 PM, Shuah Khan wrote:
>> On 8/12/19 1:10 PM, Shuah Khan wrote:
>>> On 8/12/19 12:52 PM, André Almeida wrote:
>>>> Hi Shuah,
>>>>
>>>> On 8/12/19 11:08 AM, Shuah Khan wrote:
>>>>> On 8/9/19 9:51 PM, Helen Koike wrote:
>>>>>> Hi Andre,
>>>>>>
>>>>>> Thanks for testing this.
>>>>>>
>>>>>> On 8/9/19 9:24 PM, André Almeida wrote:
>>>>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>>>>>>> Hi Andre,
>>>>>>>>
>>>>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>>>>>>> Hello Shuah,
>>>>>>>>>
>>>>>>>>> Thanks for the patch, I did some comments below.
>>>>>>>>>
>>>>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>>>>>>> vimc uses Component API to split the driver into functional
>>>>>>>>>> components.
>>>>>>>>>> The real hardware resembles a monolith structure than component and
>>>>>>>>>> component structure added a level of complexity making it hard to
>>>>>>>>>> maintain without adding any real benefit.
>>>>>>>>>>         The sensor is one vimc component that would makes sense to be a
>>>>>>>>>> separate
>>>>>>>>>> module to closely align with the real hardware. It would be easier to
>>>>>>>>>> collapse vimc into single monolithic driver first and then split the
>>>>>>>>>> sensor off as a separate module.
>>>>>>>>>>
>>>>>>>>>> This patch series emoves the component API and makes minimal
>>>>>>>>>> changes to
>>>>>>>>>> the code base preserving the functional division of the code
>>>>>>>>>> structure.
>>>>>>>>>> Preserving the functional structure allows us to split the sensor off
>>>>>>>>>> as a separate module in the future.
>>>>>>>>>>
>>>>>>>>>> Major design elements in this change are:
>>>>>>>>>>         - Use existing struct vimc_ent_config and struct
>>>>>>>>>> vimc_pipeline_config
>>>>>>>>>>           to drive the initialization of the functional components.
>>>>>>>>>>         - Make vimc_ent_config global by moving it to vimc.h
>>>>>>>>>>         - Add two new hooks add and rm to initialize and register,
>>>>>>>>>> unregister
>>>>>>>>>>           and free subdevs.
>>>>>>>>>>         - All component API is now gone and bind and unbind hooks are
>>>>>>>>>> modified
>>>>>>>>>>           to do "add" and "rm" with minimal changes to just add and rm
>>>>>>>>>> subdevs.
>>>>>>>>>>         - vimc-core's bind and unbind are now register and unregister.
>>>>>>>>>>         - vimc-core invokes "add" hooks from its
>>>>>>>>>> vimc_register_devices().
>>>>>>>>>>           The "add" hooks remain the same and register subdevs. They
>>>>>>>>>> don't
>>>>>>>>>>           create platform devices of their own and use vimc's
>>>>>>>>>> pdev.dev as
>>>>>>>>>>           their reference device. The "add" hooks save their
>>>>>>>>>> vimc_ent_device(s)
>>>>>>>>>>           in the corresponding vimc_ent_config.
>>>>>>>>>>         - vimc-core invokes "rm" hooks from its unregister to
>>>>>>>>>> unregister
>>>>>>>>>> subdevs
>>>>>>>>>>           and cleanup.
>>>>>>>>>>         - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>>>>>>>> vimc_device
>>>>>>>>>>           and the corresponding struct vimc_ent_config pointer.
>>>>>>>>>>         The following configure and stream test works on all devices.
>>>>>>>>>>              media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>         media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>         media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>         media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>              v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>>>>>>> width=1920,height=1440
>>>>>>>>>>         v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
>>>>>>>>>> pixelformat=BA81
>>>>>>>>>>         v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
>>>>>>>>>> pixelformat=BA81
>>>>>>>>>>              v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>>>>>>>         v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>>>>>>         v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>>>>>>
>>>>>>>>>> The third patch in the series fixes a general protection fault found
>>>>>>>>>> when rmmod is done while stream is active.
>>>>>>>>>
>>>>>>>>> I applied your patch on top of media_tree/master and I did some
>>>>>>>>> testing.
>>>>>>>>> Not sure if I did something wrong, but just adding and removing the
>>>>>>>>> module generated a kernel panic:
>>>>>>>>
>>>>>>>> Thanks for testing.
>>>>>>>>
>>>>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>>>>>>> I will apply these to media latest and work from there. I have to
>>>>>>>> rebase these on top of the reverts from Lucas and Helen
>>>>>>>
>>>>>>> Ok, please let me know if I succeeded to reproduce.
>>>>>>>
>>>>>>>>>
>>>>>>>>> ~# modprobe vimc
>>>>>>>>> ~# rmmod vimc
>>>>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>>>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>>
>>>>>>>>> <registers values...>
>>>>>>>>>
>>>>>>>>> [   16.469188] Call Trace:
>>>>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>>>>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>>>>>>
>>>>>>>>> <registers values...>
>>>>>>>>>
>>>>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>>
>>>>>>>>> <registers values...>
>>>>>>>>>
>>>>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
>>>>>>>>> error)
>>>>>>>>>
>>>>>>>>> I just added the module after booting, no other action was made.
>>>>>>>>> Here is
>>>>>>>>> how my `git log --oneline` looks like:
>>>>>>>>>
>>>>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>>>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>>>>>>> monolithic driver
>>>>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>>>
>>>>>> I couldn't reproduce the error, my tree looks the same:
>>>>>>
>>>>>> [I] koike@floko ~/m/o/linux> git log --oneline
>>>>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
>>>>>> active
>>>>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
>>>>>> monolithic driver
>>>>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
>>>>>> 97299a303532 (media/master) media: Remove dev_err() usage after
>>>>>> platform_get_irq()
>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>>
>>>>> Thanks Helen for trying to reproduce and sharing the result.
>>>>
>>>> Me and Helen found out what is the problem. If you follow this call trace:
>>>>
>>>> vimc_ent_sd_unregister()
>>>> v4l2_device_unregister_subdev()
>>>> v4l2_subdev_release()
>>>>
>>>> You'll notice that this last function calls the `release` callback
>>>> implementation of the subdevice. For instance, the `release` of
>>>> vimc-sensor is this one:
>>>>
>>>> static void vimc_sen_release(struct v4l2_subdev *sd)
>>>> {
>>>>      struct vimc_sen_device *vsen =
>>>>                  container_of(sd, struct vimc_sen_device, sd);
>>>>
>>>>      v4l2_ctrl_handler_free(&vsen->hdl);
>>>>      tpg_free(&vsen->tpg);
>>>>      kfree(vsen);
>>>> }
>>>>
>>>> And then you can see that `vsen` has been freed. Back to
>>>> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
>>>> function will call vimc_pads_cleanup(). This is basically a
>>>> kfree(ved->pads), but `ved` has just been freed at
>>>> v4l2_subdev_release(), producing a memory fault.
>>>>
>>>> To fix that, we found two options:
>>>>
>>>> - place the kfree(ved->pads) inside the release callback of each
>>>> subdevice and removing vimc_pads_cleanup() from
>>>> vimc_ent_sd_unregister()
>>>> - use a auxiliary variable to hold the address of the pads, for instance:
>>>>
>>>> void vimc_ent_sd_unregister(...)
>>>> {
>>>>       struct media_pad *pads = ved->pads;
>>>>       ...
>>>>       vimc_pads_cleanup(pads);
>>>> }
>>>>
>>>>
>>>
>>> I fixed a problem in the thirds patch. vimc-capture uses the first
>>> approach - placing the kfree(ved->pads) inside the release callback.
>>>
>>> I am debugging another such problem in unbind path while streaming.
>>> I am working on v2 and I will look for the rmmod problem and fix it.
>>>
>>> thanks again for testing and finding the root cause.
>>> -- Shuah
>>
>> Hi Andre,
>>
>> Here is what's happening.
>>
>> Before this change, you can't really do rmmod vimc, because vimc is in
>> use by other component drivers. With the collapse, now you can actually
>> do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees
>> pads first and the does v4l2_device_unregister_subdev().
>>
>> I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
>>
>> Did you test with the third patch in this series?
> 
> yes, we tested with 3/3, but the new problem now is when doing the following
> in this order:
> 
>      v4l2_device_unregister_subdev(sd);
>      vimc_pads_cleanup(ved->pads);
> 
> 
> v4l2_device_unregister_subdev() calls the release function of the subdevice that
> frees the ved object, so ved->pads is not valid anymore. That is why André suggested
> a temporary variable to hold ved->pads and to be able to free it later:
> 
>      struct media_pad *pads = ved->pads;
> 
>      v4l2_device_unregister_subdev(sd);
>      vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.
> 
> 

Got it.

thanks,
-- Shuah

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Laurent Pinchart]

Hi Helen,

On Mon, Aug 12, 2019 at 08:41:33PM -0300, Helen Koike wrote:
> On 8/12/19 7:14 PM, Shuah Khan wrote:
> > On 8/12/19 1:10 PM, Shuah Khan wrote:
> >> On 8/12/19 12:52 PM, André Almeida wrote:
> >>> On 8/12/19 11:08 AM, Shuah Khan wrote:
> >>>> On 8/9/19 9:51 PM, Helen Koike wrote:
> >>>>> On 8/9/19 9:24 PM, André Almeida wrote:
> >>>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
> >>>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
> >>>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
> >>>>>>>>> vimc uses Component API to split the driver into functional
> >>>>>>>>> components.
> >>>>>>>>> The real hardware resembles a monolith structure than component and
> >>>>>>>>> component structure added a level of complexity making it hard to
> >>>>>>>>> maintain without adding any real benefit.
> >>>>>>>>>        The sensor is one vimc component that would makes sense to be a
> >>>>>>>>> separate
> >>>>>>>>> module to closely align with the real hardware. It would be easier to
> >>>>>>>>> collapse vimc into single monolithic driver first and then split the
> >>>>>>>>> sensor off as a separate module.
> >>>>>>>>>
> >>>>>>>>> This patch series emoves the component API and makes minimal
> >>>>>>>>> changes to
> >>>>>>>>> the code base preserving the functional division of the code
> >>>>>>>>> structure.
> >>>>>>>>> Preserving the functional structure allows us to split the sensor off
> >>>>>>>>> as a separate module in the future.
> >>>>>>>>>
> >>>>>>>>> Major design elements in this change are:
> >>>>>>>>>        - Use existing struct vimc_ent_config and struct
> >>>>>>>>> vimc_pipeline_config
> >>>>>>>>>          to drive the initialization of the functional components.
> >>>>>>>>>        - Make vimc_ent_config global by moving it to vimc.h
> >>>>>>>>>        - Add two new hooks add and rm to initialize and register,
> >>>>>>>>> unregister
> >>>>>>>>>          and free subdevs.
> >>>>>>>>>        - All component API is now gone and bind and unbind hooks are
> >>>>>>>>> modified
> >>>>>>>>>          to do "add" and "rm" with minimal changes to just add and rm
> >>>>>>>>> subdevs.
> >>>>>>>>>        - vimc-core's bind and unbind are now register and unregister.
> >>>>>>>>>        - vimc-core invokes "add" hooks from its
> >>>>>>>>> vimc_register_devices().
> >>>>>>>>>          The "add" hooks remain the same and register subdevs. They
> >>>>>>>>> don't
> >>>>>>>>>          create platform devices of their own and use vimc's
> >>>>>>>>> pdev.dev as
> >>>>>>>>>          their reference device. The "add" hooks save their
> >>>>>>>>> vimc_ent_device(s)
> >>>>>>>>>          in the corresponding vimc_ent_config.
> >>>>>>>>>        - vimc-core invokes "rm" hooks from its unregister to
> >>>>>>>>> unregister
> >>>>>>>>> subdevs
> >>>>>>>>>          and cleanup.
> >>>>>>>>>        - vimc-core invokes "add" and "rm" hooks with pointer to struct
> >>>>>>>>> vimc_device
> >>>>>>>>>          and the corresponding struct vimc_ent_config pointer.
> >>>>>>>>>        The following configure and stream test works on all devices.
> >>>>>>>>>             media-ctl -d platform:vimc -V '"Sensor
> >>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
> >>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>        media-ctl -d platform:vimc -V '"Sensor
> >>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
> >>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>             v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
> >>>>>>>>> width=1920,height=1440
> >>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
> >>>>>>>>> pixelformat=BA81
> >>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
> >>>>>>>>> pixelformat=BA81
> >>>>>>>>>             v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
> >>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
> >>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
> >>>>>>>>>
> >>>>>>>>> The third patch in the series fixes a general protection fault found
> >>>>>>>>> when rmmod is done while stream is active.
> >>>>>>>>
> >>>>>>>> I applied your patch on top of media_tree/master and I did some
> >>>>>>>> testing.
> >>>>>>>> Not sure if I did something wrong, but just adding and removing the
> >>>>>>>> module generated a kernel panic:
> >>>>>>>
> >>>>>>> Thanks for testing.
> >>>>>>>
> >>>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
> >>>>>>> I will apply these to media latest and work from there. I have to
> >>>>>>> rebase these on top of the reverts from Lucas and Helen
> >>>>>>
> >>>>>> Ok, please let me know if I succeeded to reproduce.
> >>>>>>
> >>>>>>>>
> >>>>>>>> ~# modprobe vimc
> >>>>>>>> ~# rmmod vimc
> >>>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
> >>>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
> >>>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> >>>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
> >>>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
> >>>>>>>>
> >>>>>>>> <registers values...>
> >>>>>>>>
> >>>>>>>> [   16.469188] Call Trace:
> >>>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
> >>>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
> >>>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
> >>>>>>>> [   16.472184]  driver_detach+0x37/0x6b
> >>>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
> >>>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
> >>>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
> >>>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
> >>>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> >>>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
> >>>>>>>>
> >>>>>>>> <registers values...>
> >>>>>>>>
> >>>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
> >>>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
> >>>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
> >>>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
> >>>>>>>>
> >>>>>>>> <registers values...>
> >>>>>>>>
> >>>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
> >>>>>>>> error)
> >>>>>>>>
> >>>>>>>> I just added the module after booting, no other action was made.
> >>>>>>>> Here is
> >>>>>>>> how my `git log --oneline` looks like:
> >>>>>>>>
> >>>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
> >>>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
> >>>>>>>> monolithic driver
> >>>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
> >>>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
> >>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
> >>>>>
> >>>>> I couldn't reproduce the error, my tree looks the same:
> >>>>>
> >>>>> [I] koike@floko ~/m/o/linux> git log --oneline
> >>>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
> >>>>> active
> >>>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
> >>>>> monolithic driver
> >>>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
> >>>>> 97299a303532 (media/master) media: Remove dev_err() usage after
> >>>>> platform_get_irq()
> >>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
> >>>>
> >>>> Thanks Helen for trying to reproduce and sharing the result.
> >>>
> >>> Me and Helen found out what is the problem. If you follow this call trace:
> >>>
> >>> vimc_ent_sd_unregister()
> >>> v4l2_device_unregister_subdev()
> >>> v4l2_subdev_release()
> >>>
> >>> You'll notice that this last function calls the `release` callback
> >>> implementation of the subdevice. For instance, the `release` of
> >>> vimc-sensor is this one:
> >>>
> >>> static void vimc_sen_release(struct v4l2_subdev *sd)
> >>> {
> >>>     struct vimc_sen_device *vsen =
> >>>                 container_of(sd, struct vimc_sen_device, sd);
> >>>
> >>>     v4l2_ctrl_handler_free(&vsen->hdl);
> >>>     tpg_free(&vsen->tpg);
> >>>     kfree(vsen);
> >>> }
> >>>
> >>> And then you can see that `vsen` has been freed. Back to
> >>> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
> >>> function will call vimc_pads_cleanup(). This is basically a
> >>> kfree(ved->pads), but `ved` has just been freed at
> >>> v4l2_subdev_release(), producing a memory fault.
> >>>
> >>> To fix that, we found two options:
> >>>
> >>> - place the kfree(ved->pads) inside the release callback of each
> >>> subdevice and removing vimc_pads_cleanup() from
> >>> vimc_ent_sd_unregister()
> >>> - use a auxiliary variable to hold the address of the pads, for instance:
> >>>
> >>> void vimc_ent_sd_unregister(...)
> >>> {
> >>>      struct media_pad *pads = ved->pads;
> >>>      ...
> >>>      vimc_pads_cleanup(pads);
> >>> }
> >>>
> >>>
> >>
> >> I fixed a problem in the thirds patch. vimc-capture uses the first
> >> approach - placing the kfree(ved->pads) inside the release callback.
> >>
> >> I am debugging another such problem in unbind path while streaming.
> >> I am working on v2 and I will look for the rmmod problem and fix it.
> >>
> >> thanks again for testing and finding the root cause.
> >> -- Shuah
> > 
> > Hi Andre,
> > 
> > Here is what's happening.
> > 
> > Before this change, you can't really do rmmod vimc, because vimc is in
> > use by other component drivers. With the collapse, now you can actually
> > do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees
> > pads first and the does v4l2_device_unregister_subdev().
> > 
> > I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
> > 
> > Did you test with the third patch in this series?
> 
> yes, we tested with 3/3, but the new problem now is when doing the following
> in this order:
> 
>     v4l2_device_unregister_subdev(sd);
>     vimc_pads_cleanup(ved->pads);
> 
> 
> v4l2_device_unregister_subdev() calls the release function of the subdevice that
> frees the ved object, so ved->pads is not valid anymore. That is why André suggested
> a temporary variable to hold ved->pads and to be able to free it later:
> 
>     struct media_pad *pads = ved->pads;
> 
>     v4l2_device_unregister_subdev(sd);
>     vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.

Can't you simply call vimc_pads_cleanup() in the release function of the
subdevice before freeing the ved object ?

-- 
Regards,

Laurent Pinchart

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Helen Koike]

On 8/13/19 6:56 AM, Laurent Pinchart wrote:
> Hi Helen,
> 
> On Mon, Aug 12, 2019 at 08:41:33PM -0300, Helen Koike wrote:
>> On 8/12/19 7:14 PM, Shuah Khan wrote:
>>> On 8/12/19 1:10 PM, Shuah Khan wrote:
>>>> On 8/12/19 12:52 PM, André Almeida wrote:
>>>>> On 8/12/19 11:08 AM, Shuah Khan wrote:
>>>>>> On 8/9/19 9:51 PM, Helen Koike wrote:
>>>>>>> On 8/9/19 9:24 PM, André Almeida wrote:
>>>>>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>>>>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>>>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>>>>>>>> vimc uses Component API to split the driver into functional
>>>>>>>>>>> components.
>>>>>>>>>>> The real hardware resembles a monolith structure than component and
>>>>>>>>>>> component structure added a level of complexity making it hard to
>>>>>>>>>>> maintain without adding any real benefit.
>>>>>>>>>>>        The sensor is one vimc component that would makes sense to be a
>>>>>>>>>>> separate
>>>>>>>>>>> module to closely align with the real hardware. It would be easier to
>>>>>>>>>>> collapse vimc into single monolithic driver first and then split the
>>>>>>>>>>> sensor off as a separate module.
>>>>>>>>>>>
>>>>>>>>>>> This patch series emoves the component API and makes minimal
>>>>>>>>>>> changes to
>>>>>>>>>>> the code base preserving the functional division of the code
>>>>>>>>>>> structure.
>>>>>>>>>>> Preserving the functional structure allows us to split the sensor off
>>>>>>>>>>> as a separate module in the future.
>>>>>>>>>>>
>>>>>>>>>>> Major design elements in this change are:
>>>>>>>>>>>        - Use existing struct vimc_ent_config and struct
>>>>>>>>>>> vimc_pipeline_config
>>>>>>>>>>>          to drive the initialization of the functional components.
>>>>>>>>>>>        - Make vimc_ent_config global by moving it to vimc.h
>>>>>>>>>>>        - Add two new hooks add and rm to initialize and register,
>>>>>>>>>>> unregister
>>>>>>>>>>>          and free subdevs.
>>>>>>>>>>>        - All component API is now gone and bind and unbind hooks are
>>>>>>>>>>> modified
>>>>>>>>>>>          to do "add" and "rm" with minimal changes to just add and rm
>>>>>>>>>>> subdevs.
>>>>>>>>>>>        - vimc-core's bind and unbind are now register and unregister.
>>>>>>>>>>>        - vimc-core invokes "add" hooks from its
>>>>>>>>>>> vimc_register_devices().
>>>>>>>>>>>          The "add" hooks remain the same and register subdevs. They
>>>>>>>>>>> don't
>>>>>>>>>>>          create platform devices of their own and use vimc's
>>>>>>>>>>> pdev.dev as
>>>>>>>>>>>          their reference device. The "add" hooks save their
>>>>>>>>>>> vimc_ent_device(s)
>>>>>>>>>>>          in the corresponding vimc_ent_config.
>>>>>>>>>>>        - vimc-core invokes "rm" hooks from its unregister to
>>>>>>>>>>> unregister
>>>>>>>>>>> subdevs
>>>>>>>>>>>          and cleanup.
>>>>>>>>>>>        - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>>>>>>>>> vimc_device
>>>>>>>>>>>          and the corresponding struct vimc_ent_config pointer.
>>>>>>>>>>>        The following configure and stream test works on all devices.
>>>>>>>>>>>             media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>        media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>             v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>>>>>>>> width=1920,height=1440
>>>>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
>>>>>>>>>>> pixelformat=BA81
>>>>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
>>>>>>>>>>> pixelformat=BA81
>>>>>>>>>>>             v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>>>>>>>
>>>>>>>>>>> The third patch in the series fixes a general protection fault found
>>>>>>>>>>> when rmmod is done while stream is active.
>>>>>>>>>>
>>>>>>>>>> I applied your patch on top of media_tree/master and I did some
>>>>>>>>>> testing.
>>>>>>>>>> Not sure if I did something wrong, but just adding and removing the
>>>>>>>>>> module generated a kernel panic:
>>>>>>>>>
>>>>>>>>> Thanks for testing.
>>>>>>>>>
>>>>>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>>>>>>>> I will apply these to media latest and work from there. I have to
>>>>>>>>> rebase these on top of the reverts from Lucas and Helen
>>>>>>>>
>>>>>>>> Ok, please let me know if I succeeded to reproduce.
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ~# modprobe vimc
>>>>>>>>>> ~# rmmod vimc
>>>>>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>>>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>>>>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>>>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>>>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>>>
>>>>>>>>>> <registers values...>
>>>>>>>>>>
>>>>>>>>>> [   16.469188] Call Trace:
>>>>>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>>>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>>>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>>>>>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>>>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>>>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>>>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>>>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>>>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>>>>>>>
>>>>>>>>>> <registers values...>
>>>>>>>>>>
>>>>>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>>>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>>>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>>>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>>>
>>>>>>>>>> <registers values...>
>>>>>>>>>>
>>>>>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
>>>>>>>>>> error)
>>>>>>>>>>
>>>>>>>>>> I just added the module after booting, no other action was made.
>>>>>>>>>> Here is
>>>>>>>>>> how my `git log --oneline` looks like:
>>>>>>>>>>
>>>>>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>>>>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>>>>>>>> monolithic driver
>>>>>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>>>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>>>>
>>>>>>> I couldn't reproduce the error, my tree looks the same:
>>>>>>>
>>>>>>> [I] koike@floko ~/m/o/linux> git log --oneline
>>>>>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
>>>>>>> active
>>>>>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
>>>>>>> monolithic driver
>>>>>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
>>>>>>> 97299a303532 (media/master) media: Remove dev_err() usage after
>>>>>>> platform_get_irq()
>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>>>
>>>>>> Thanks Helen for trying to reproduce and sharing the result.
>>>>>
>>>>> Me and Helen found out what is the problem. If you follow this call trace:
>>>>>
>>>>> vimc_ent_sd_unregister()
>>>>> v4l2_device_unregister_subdev()
>>>>> v4l2_subdev_release()
>>>>>
>>>>> You'll notice that this last function calls the `release` callback
>>>>> implementation of the subdevice. For instance, the `release` of
>>>>> vimc-sensor is this one:
>>>>>
>>>>> static void vimc_sen_release(struct v4l2_subdev *sd)
>>>>> {
>>>>>     struct vimc_sen_device *vsen =
>>>>>                 container_of(sd, struct vimc_sen_device, sd);
>>>>>
>>>>>     v4l2_ctrl_handler_free(&vsen->hdl);
>>>>>     tpg_free(&vsen->tpg);
>>>>>     kfree(vsen);
>>>>> }
>>>>>
>>>>> And then you can see that `vsen` has been freed. Back to
>>>>> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
>>>>> function will call vimc_pads_cleanup(). This is basically a
>>>>> kfree(ved->pads), but `ved` has just been freed at
>>>>> v4l2_subdev_release(), producing a memory fault.
>>>>>
>>>>> To fix that, we found two options:
>>>>>
>>>>> - place the kfree(ved->pads) inside the release callback of each
>>>>> subdevice and removing vimc_pads_cleanup() from
>>>>> vimc_ent_sd_unregister()
>>>>> - use a auxiliary variable to hold the address of the pads, for instance:
>>>>>
>>>>> void vimc_ent_sd_unregister(...)
>>>>> {
>>>>>      struct media_pad *pads = ved->pads;
>>>>>      ...
>>>>>      vimc_pads_cleanup(pads);
>>>>> }
>>>>>
>>>>>
>>>>
>>>> I fixed a problem in the thirds patch. vimc-capture uses the first
>>>> approach - placing the kfree(ved->pads) inside the release callback.
>>>>
>>>> I am debugging another such problem in unbind path while streaming.
>>>> I am working on v2 and I will look for the rmmod problem and fix it.
>>>>
>>>> thanks again for testing and finding the root cause.
>>>> -- Shuah
>>>
>>> Hi Andre,
>>>
>>> Here is what's happening.
>>>
>>> Before this change, you can't really do rmmod vimc, because vimc is in
>>> use by other component drivers. With the collapse, now you can actually
>>> do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees
>>> pads first and the does v4l2_device_unregister_subdev().
>>>
>>> I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
>>>
>>> Did you test with the third patch in this series?
>>
>> yes, we tested with 3/3, but the new problem now is when doing the following
>> in this order:
>>
>>     v4l2_device_unregister_subdev(sd);
>>     vimc_pads_cleanup(ved->pads);
>>
>>
>> v4l2_device_unregister_subdev() calls the release function of the subdevice that
>> frees the ved object, so ved->pads is not valid anymore. That is why André suggested
>> a temporary variable to hold ved->pads and to be able to free it later:
>>
>>     struct media_pad *pads = ved->pads;
>>
>>     v4l2_device_unregister_subdev(sd);
>>     vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.
> 
> Can't you simply call vimc_pads_cleanup() in the release function of the
> subdevice before freeing the ved object ?
> 

Yes we can, that is the other option Andre suggested.
The  advantage of doing it inside vimc_ent_sd_unregister() is that
who allocated the memory in the first place was vimc_ent_sd_register(), and also,
this is a common code to all subdevs, so letting it in vimc_ent_sd_unregister()
will make sure no subdevs ever forget to free this memory.
But saving the pointer to a variable to free it later is not that pretty
either.

Helen

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Laurent Pinchart]

Hi Helen,

On Tue, Aug 13, 2019 at 09:25:01AM -0300, Helen Koike wrote:
> On 8/13/19 6:56 AM, Laurent Pinchart wrote:
> > On Mon, Aug 12, 2019 at 08:41:33PM -0300, Helen Koike wrote:
> >> On 8/12/19 7:14 PM, Shuah Khan wrote:
> >>> On 8/12/19 1:10 PM, Shuah Khan wrote:
> >>>> On 8/12/19 12:52 PM, André Almeida wrote:
> >>>>> On 8/12/19 11:08 AM, Shuah Khan wrote:
> >>>>>> On 8/9/19 9:51 PM, Helen Koike wrote:
> >>>>>>> On 8/9/19 9:24 PM, André Almeida wrote:
> >>>>>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
> >>>>>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
> >>>>>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
> >>>>>>>>>>> vimc uses Component API to split the driver into functional
> >>>>>>>>>>> components.
> >>>>>>>>>>> The real hardware resembles a monolith structure than component and
> >>>>>>>>>>> component structure added a level of complexity making it hard to
> >>>>>>>>>>> maintain without adding any real benefit.
> >>>>>>>>>>>        The sensor is one vimc component that would makes sense to be a
> >>>>>>>>>>> separate
> >>>>>>>>>>> module to closely align with the real hardware. It would be easier to
> >>>>>>>>>>> collapse vimc into single monolithic driver first and then split the
> >>>>>>>>>>> sensor off as a separate module.
> >>>>>>>>>>>
> >>>>>>>>>>> This patch series emoves the component API and makes minimal
> >>>>>>>>>>> changes to
> >>>>>>>>>>> the code base preserving the functional division of the code
> >>>>>>>>>>> structure.
> >>>>>>>>>>> Preserving the functional structure allows us to split the sensor off
> >>>>>>>>>>> as a separate module in the future.
> >>>>>>>>>>>
> >>>>>>>>>>> Major design elements in this change are:
> >>>>>>>>>>>        - Use existing struct vimc_ent_config and struct
> >>>>>>>>>>> vimc_pipeline_config
> >>>>>>>>>>>          to drive the initialization of the functional components.
> >>>>>>>>>>>        - Make vimc_ent_config global by moving it to vimc.h
> >>>>>>>>>>>        - Add two new hooks add and rm to initialize and register,
> >>>>>>>>>>> unregister
> >>>>>>>>>>>          and free subdevs.
> >>>>>>>>>>>        - All component API is now gone and bind and unbind hooks are
> >>>>>>>>>>> modified
> >>>>>>>>>>>          to do "add" and "rm" with minimal changes to just add and rm
> >>>>>>>>>>> subdevs.
> >>>>>>>>>>>        - vimc-core's bind and unbind are now register and unregister.
> >>>>>>>>>>>        - vimc-core invokes "add" hooks from its
> >>>>>>>>>>> vimc_register_devices().
> >>>>>>>>>>>          The "add" hooks remain the same and register subdevs. They
> >>>>>>>>>>> don't
> >>>>>>>>>>>          create platform devices of their own and use vimc's
> >>>>>>>>>>> pdev.dev as
> >>>>>>>>>>>          their reference device. The "add" hooks save their
> >>>>>>>>>>> vimc_ent_device(s)
> >>>>>>>>>>>          in the corresponding vimc_ent_config.
> >>>>>>>>>>>        - vimc-core invokes "rm" hooks from its unregister to
> >>>>>>>>>>> unregister
> >>>>>>>>>>> subdevs
> >>>>>>>>>>>          and cleanup.
> >>>>>>>>>>>        - vimc-core invokes "add" and "rm" hooks with pointer to struct
> >>>>>>>>>>> vimc_device
> >>>>>>>>>>>          and the corresponding struct vimc_ent_config pointer.
> >>>>>>>>>>>        The following configure and stream test works on all devices.
> >>>>>>>>>>>             media-ctl -d platform:vimc -V '"Sensor
> >>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
> >>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>>>        media-ctl -d platform:vimc -V '"Sensor
> >>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>>>        media-ctl -d platform:vimc -V '"Debayer
> >>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
> >>>>>>>>>>>             v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
> >>>>>>>>>>> width=1920,height=1440
> >>>>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
> >>>>>>>>>>> pixelformat=BA81
> >>>>>>>>>>>        v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
> >>>>>>>>>>> pixelformat=BA81
> >>>>>>>>>>>             v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
> >>>>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
> >>>>>>>>>>>        v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
> >>>>>>>>>>>
> >>>>>>>>>>> The third patch in the series fixes a general protection fault found
> >>>>>>>>>>> when rmmod is done while stream is active.
> >>>>>>>>>>
> >>>>>>>>>> I applied your patch on top of media_tree/master and I did some
> >>>>>>>>>> testing.
> >>>>>>>>>> Not sure if I did something wrong, but just adding and removing the
> >>>>>>>>>> module generated a kernel panic:
> >>>>>>>>>
> >>>>>>>>> Thanks for testing.
> >>>>>>>>>
> >>>>>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
> >>>>>>>>> I will apply these to media latest and work from there. I have to
> >>>>>>>>> rebase these on top of the reverts from Lucas and Helen
> >>>>>>>>
> >>>>>>>> Ok, please let me know if I succeeded to reproduce.
> >>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> ~# modprobe vimc
> >>>>>>>>>> ~# rmmod vimc
> >>>>>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
> >>>>>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
> >>>>>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> >>>>>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
> >>>>>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
> >>>>>>>>>>
> >>>>>>>>>> <registers values...>
> >>>>>>>>>>
> >>>>>>>>>> [   16.469188] Call Trace:
> >>>>>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
> >>>>>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
> >>>>>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
> >>>>>>>>>> [   16.472184]  driver_detach+0x37/0x6b
> >>>>>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
> >>>>>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
> >>>>>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
> >>>>>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
> >>>>>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
> >>>>>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
> >>>>>>>>>>
> >>>>>>>>>> <registers values...>
> >>>>>>>>>>
> >>>>>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
> >>>>>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
> >>>>>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
> >>>>>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
> >>>>>>>>>>
> >>>>>>>>>> <registers values...>
> >>>>>>>>>>
> >>>>>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
> >>>>>>>>>> error)
> >>>>>>>>>>
> >>>>>>>>>> I just added the module after booting, no other action was made.
> >>>>>>>>>> Here is
> >>>>>>>>>> how my `git log --oneline` looks like:
> >>>>>>>>>>
> >>>>>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
> >>>>>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
> >>>>>>>>>> monolithic driver
> >>>>>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
> >>>>>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
> >>>>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
> >>>>>>>
> >>>>>>> I couldn't reproduce the error, my tree looks the same:
> >>>>>>>
> >>>>>>> [I] koike@floko ~/m/o/linux> git log --oneline
> >>>>>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
> >>>>>>> active
> >>>>>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
> >>>>>>> monolithic driver
> >>>>>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
> >>>>>>> 97299a303532 (media/master) media: Remove dev_err() usage after
> >>>>>>> platform_get_irq()
> >>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
> >>>>>>
> >>>>>> Thanks Helen for trying to reproduce and sharing the result.
> >>>>>
> >>>>> Me and Helen found out what is the problem. If you follow this call trace:
> >>>>>
> >>>>> vimc_ent_sd_unregister()
> >>>>> v4l2_device_unregister_subdev()
> >>>>> v4l2_subdev_release()
> >>>>>
> >>>>> You'll notice that this last function calls the `release` callback
> >>>>> implementation of the subdevice. For instance, the `release` of
> >>>>> vimc-sensor is this one:
> >>>>>
> >>>>> static void vimc_sen_release(struct v4l2_subdev *sd)
> >>>>> {
> >>>>>     struct vimc_sen_device *vsen =
> >>>>>                 container_of(sd, struct vimc_sen_device, sd);
> >>>>>
> >>>>>     v4l2_ctrl_handler_free(&vsen->hdl);
> >>>>>     tpg_free(&vsen->tpg);
> >>>>>     kfree(vsen);
> >>>>> }
> >>>>>
> >>>>> And then you can see that `vsen` has been freed. Back to
> >>>>> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
> >>>>> function will call vimc_pads_cleanup(). This is basically a
> >>>>> kfree(ved->pads), but `ved` has just been freed at
> >>>>> v4l2_subdev_release(), producing a memory fault.
> >>>>>
> >>>>> To fix that, we found two options:
> >>>>>
> >>>>> - place the kfree(ved->pads) inside the release callback of each
> >>>>> subdevice and removing vimc_pads_cleanup() from
> >>>>> vimc_ent_sd_unregister()
> >>>>> - use a auxiliary variable to hold the address of the pads, for instance:
> >>>>>
> >>>>> void vimc_ent_sd_unregister(...)
> >>>>> {
> >>>>>      struct media_pad *pads = ved->pads;
> >>>>>      ...
> >>>>>      vimc_pads_cleanup(pads);
> >>>>> }
> >>>>>
> >>>>>
> >>>>
> >>>> I fixed a problem in the thirds patch. vimc-capture uses the first
> >>>> approach - placing the kfree(ved->pads) inside the release callback.
> >>>>
> >>>> I am debugging another such problem in unbind path while streaming.
> >>>> I am working on v2 and I will look for the rmmod problem and fix it.
> >>>>
> >>>> thanks again for testing and finding the root cause.
> >>>> -- Shuah
> >>>
> >>> Hi Andre,
> >>>
> >>> Here is what's happening.
> >>>
> >>> Before this change, you can't really do rmmod vimc, because vimc is in
> >>> use by other component drivers. With the collapse, now you can actually
> >>> do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees
> >>> pads first and the does v4l2_device_unregister_subdev().
> >>>
> >>> I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
> >>>
> >>> Did you test with the third patch in this series?
> >>
> >> yes, we tested with 3/3, but the new problem now is when doing the following
> >> in this order:
> >>
> >>     v4l2_device_unregister_subdev(sd);
> >>     vimc_pads_cleanup(ved->pads);
> >>
> >>
> >> v4l2_device_unregister_subdev() calls the release function of the subdevice that
> >> frees the ved object, so ved->pads is not valid anymore. That is why André suggested
> >> a temporary variable to hold ved->pads and to be able to free it later:
> >>
> >>     struct media_pad *pads = ved->pads;
> >>
> >>     v4l2_device_unregister_subdev(sd);
> >>     vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.
> > 
> > Can't you simply call vimc_pads_cleanup() in the release function of the
> > subdevice before freeing the ved object ?
> 
> Yes we can, that is the other option Andre suggested.
> The  advantage of doing it inside vimc_ent_sd_unregister() is that
> who allocated the memory in the first place was vimc_ent_sd_register(), and also,
> this is a common code to all subdevs, so letting it in vimc_ent_sd_unregister()
> will make sure no subdevs ever forget to free this memory.
> But saving the pointer to a variable to free it later is not that pretty
> either.

The release function of a subdevice is meant to be used as a destructor,
it should free all the resources allocated for the subdevice. Your
subdevices (such as vimc_sca_device) are essentially objects derived
from a common vimc subdevice object (vimc_ent_device), I thus suggest
providing four functions to deal with vimc_ent_device:

- vimc_ent_sd_init() to initialise the base vimc_ent_device object (most
  of the code is currently in vimc_ent_sd_register()), to be called by
  the init function of each subdevice type (the bind functions in the
  current code)

- vimc_ent_sd_register() to only register the vimc_ent_device once it
  has been fully initialised

- vimc_end_sd_unregister() to unregister the vimc_ent_device

- vimc_end_sd_cleanup() to free the resources allocated by
  vimc_ent_sd_init() (including calling vimc_pads_cleanup()), to be
  called by the subdevice release handler

-- 
Regards,

Laurent Pinchart

Re: [PATCH 0/3] Collapse vimc into single monolithic driver

[Shuah Khan]

On 8/13/19 6:36 AM, Laurent Pinchart wrote:
> Hi Helen,
> 
> On Tue, Aug 13, 2019 at 09:25:01AM -0300, Helen Koike wrote:
>> On 8/13/19 6:56 AM, Laurent Pinchart wrote:
>>> On Mon, Aug 12, 2019 at 08:41:33PM -0300, Helen Koike wrote:
>>>> On 8/12/19 7:14 PM, Shuah Khan wrote:
>>>>> On 8/12/19 1:10 PM, Shuah Khan wrote:
>>>>>> On 8/12/19 12:52 PM, André Almeida wrote:
>>>>>>> On 8/12/19 11:08 AM, Shuah Khan wrote:
>>>>>>>> On 8/9/19 9:51 PM, Helen Koike wrote:
>>>>>>>>> On 8/9/19 9:24 PM, André Almeida wrote:
>>>>>>>>>> On 8/9/19 9:17 PM, Shuah Khan wrote:
>>>>>>>>>>> On 8/9/19 5:52 PM, André Almeida wrote:
>>>>>>>>>>>> On 8/9/19 6:45 PM, Shuah Khan wrote:
>>>>>>>>>>>>> vimc uses Component API to split the driver into functional
>>>>>>>>>>>>> components.
>>>>>>>>>>>>> The real hardware resembles a monolith structure than component and
>>>>>>>>>>>>> component structure added a level of complexity making it hard to
>>>>>>>>>>>>> maintain without adding any real benefit.
>>>>>>>>>>>>>         The sensor is one vimc component that would makes sense to be a
>>>>>>>>>>>>> separate
>>>>>>>>>>>>> module to closely align with the real hardware. It would be easier to
>>>>>>>>>>>>> collapse vimc into single monolithic driver first and then split the
>>>>>>>>>>>>> sensor off as a separate module.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This patch series emoves the component API and makes minimal
>>>>>>>>>>>>> changes to
>>>>>>>>>>>>> the code base preserving the functional division of the code
>>>>>>>>>>>>> structure.
>>>>>>>>>>>>> Preserving the functional structure allows us to split the sensor off
>>>>>>>>>>>>> as a separate module in the future.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Major design elements in this change are:
>>>>>>>>>>>>>         - Use existing struct vimc_ent_config and struct
>>>>>>>>>>>>> vimc_pipeline_config
>>>>>>>>>>>>>           to drive the initialization of the functional components.
>>>>>>>>>>>>>         - Make vimc_ent_config global by moving it to vimc.h
>>>>>>>>>>>>>         - Add two new hooks add and rm to initialize and register,
>>>>>>>>>>>>> unregister
>>>>>>>>>>>>>           and free subdevs.
>>>>>>>>>>>>>         - All component API is now gone and bind and unbind hooks are
>>>>>>>>>>>>> modified
>>>>>>>>>>>>>           to do "add" and "rm" with minimal changes to just add and rm
>>>>>>>>>>>>> subdevs.
>>>>>>>>>>>>>         - vimc-core's bind and unbind are now register and unregister.
>>>>>>>>>>>>>         - vimc-core invokes "add" hooks from its
>>>>>>>>>>>>> vimc_register_devices().
>>>>>>>>>>>>>           The "add" hooks remain the same and register subdevs. They
>>>>>>>>>>>>> don't
>>>>>>>>>>>>>           create platform devices of their own and use vimc's
>>>>>>>>>>>>> pdev.dev as
>>>>>>>>>>>>>           their reference device. The "add" hooks save their
>>>>>>>>>>>>> vimc_ent_device(s)
>>>>>>>>>>>>>           in the corresponding vimc_ent_config.
>>>>>>>>>>>>>         - vimc-core invokes "rm" hooks from its unregister to
>>>>>>>>>>>>> unregister
>>>>>>>>>>>>> subdevs
>>>>>>>>>>>>>           and cleanup.
>>>>>>>>>>>>>         - vimc-core invokes "add" and "rm" hooks with pointer to struct
>>>>>>>>>>>>> vimc_device
>>>>>>>>>>>>>           and the corresponding struct vimc_ent_config pointer.
>>>>>>>>>>>>>         The following configure and stream test works on all devices.
>>>>>>>>>>>>>              media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>>>         media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>>>>>> A":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>>>         media-ctl -d platform:vimc -V '"Sensor
>>>>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>>>         media-ctl -d platform:vimc -V '"Debayer
>>>>>>>>>>>>> B":0[fmt:SBGGR8_1X8/640x480]'
>>>>>>>>>>>>>              v4l2-ctl -z platform:vimc -d "RGB/YUV Capture" -v
>>>>>>>>>>>>> width=1920,height=1440
>>>>>>>>>>>>>         v4l2-ctl -z platform:vimc -d "Raw Capture 0" -v
>>>>>>>>>>>>> pixelformat=BA81
>>>>>>>>>>>>>         v4l2-ctl -z platform:vimc -d "Raw Capture 1" -v
>>>>>>>>>>>>> pixelformat=BA81
>>>>>>>>>>>>>              v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video1
>>>>>>>>>>>>>         v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video2
>>>>>>>>>>>>>         v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video3
>>>>>>>>>>>>>
>>>>>>>>>>>>> The third patch in the series fixes a general protection fault found
>>>>>>>>>>>>> when rmmod is done while stream is active.
>>>>>>>>>>>>
>>>>>>>>>>>> I applied your patch on top of media_tree/master and I did some
>>>>>>>>>>>> testing.
>>>>>>>>>>>> Not sure if I did something wrong, but just adding and removing the
>>>>>>>>>>>> module generated a kernel panic:
>>>>>>>>>>>
>>>>>>>>>>> Thanks for testing.
>>>>>>>>>>>
>>>>>>>>>>> Odd. I tested modprobe and rmmod both.I was working on Linux 5.3-rc2.
>>>>>>>>>>> I will apply these to media latest and work from there. I have to
>>>>>>>>>>> rebase these on top of the reverts from Lucas and Helen
>>>>>>>>>>
>>>>>>>>>> Ok, please let me know if I succeeded to reproduce.
>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ~# modprobe vimc
>>>>>>>>>>>> ~# rmmod vimc
>>>>>>>>>>>> [   16.452974] stack segment: 0000 [#1] SMP PTI
>>>>>>>>>>>> [   16.453688] CPU: 0 PID: 2038 Comm: rmmod Not tainted 5.3.0-rc2+ #36
>>>>>>>>>>>> [   16.454678] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>>>>>>>>>> BIOS 1.12.0-20181126_142135-anatol 04/01/2014
>>>>>>>>>>>> [   16.456191] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>>>>>
>>>>>>>>>>>> <registers values...>
>>>>>>>>>>>>
>>>>>>>>>>>> [   16.469188] Call Trace:
>>>>>>>>>>>> [   16.469666]  vimc_remove+0x35/0x90 [vimc]
>>>>>>>>>>>> [   16.470436]  platform_drv_remove+0x1f/0x40
>>>>>>>>>>>> [   16.471233]  device_release_driver_internal+0xd3/0x1b0
>>>>>>>>>>>> [   16.472184]  driver_detach+0x37/0x6b
>>>>>>>>>>>> [   16.472882]  bus_remove_driver+0x50/0xc1
>>>>>>>>>>>> [   16.473569]  vimc_exit+0xc/0xca0 [vimc]
>>>>>>>>>>>> [   16.474231]  __x64_sys_delete_module+0x18d/0x240
>>>>>>>>>>>> [   16.475036]  do_syscall_64+0x43/0x110
>>>>>>>>>>>> [   16.475656]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>>>>>>>>>> [   16.476504] RIP: 0033:0x7fceb8dafa4b
>>>>>>>>>>>>
>>>>>>>>>>>> <registers values...>
>>>>>>>>>>>>
>>>>>>>>>>>> [   16.484853] Modules linked in: vimc(-) videobuf2_vmalloc
>>>>>>>>>>>> videobuf2_memops v4l2_tpg videobuf2_v4l2 videobuf2_common
>>>>>>>>>>>> [   16.486187] ---[ end trace 91e5e0894e254d49 ]---
>>>>>>>>>>>> [   16.486758] RIP: 0010:kfree+0x4d/0x240
>>>>>>>>>>>>
>>>>>>>>>>>> <registers values...>
>>>>>>>>>>>>
>>>>>>>>>>>> fish: “rmmod vimc” terminated by signal SIGSEGV (Address boundary
>>>>>>>>>>>> error)
>>>>>>>>>>>>
>>>>>>>>>>>> I just added the module after booting, no other action was made.
>>>>>>>>>>>> Here is
>>>>>>>>>>>> how my `git log --oneline` looks like:
>>>>>>>>>>>>
>>>>>>>>>>>> 897d708e922b media: vimc: Fix gpf in rmmod path when stream is active
>>>>>>>>>>>> 2e4a5ad8ad6d media: vimc: Collapse component structure into a single
>>>>>>>>>>>> monolithic driver
>>>>>>>>>>>> 7c8da1687e92 media: vimc: move private defines to a common header
>>>>>>>>>>>> 97299a303532 media: Remove dev_err() usage after platform_get_irq()
>>>>>>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>>>>>>
>>>>>>>>> I couldn't reproduce the error, my tree looks the same:
>>>>>>>>>
>>>>>>>>> [I] koike@floko ~/m/o/linux> git log --oneline
>>>>>>>>> e3345155c8ed (HEAD) media: vimc: Fix gpf in rmmod path when stream is
>>>>>>>>> active
>>>>>>>>> 43e9e2fe761f media: vimc: Collapse component structure into a single
>>>>>>>>> monolithic driver
>>>>>>>>> 8a6d0b9adde0 media: vimc: move private defines to a common header
>>>>>>>>> 97299a303532 (media/master) media: Remove dev_err() usage after
>>>>>>>>> platform_get_irq()
>>>>>>>>> 25a3d6bac6b9 media: adv7511/cobalt: rename driver name to adv7511-v4l2
>>>>>>>>
>>>>>>>> Thanks Helen for trying to reproduce and sharing the result.
>>>>>>>
>>>>>>> Me and Helen found out what is the problem. If you follow this call trace:
>>>>>>>
>>>>>>> vimc_ent_sd_unregister()
>>>>>>> v4l2_device_unregister_subdev()
>>>>>>> v4l2_subdev_release()
>>>>>>>
>>>>>>> You'll notice that this last function calls the `release` callback
>>>>>>> implementation of the subdevice. For instance, the `release` of
>>>>>>> vimc-sensor is this one:
>>>>>>>
>>>>>>> static void vimc_sen_release(struct v4l2_subdev *sd)
>>>>>>> {
>>>>>>>      struct vimc_sen_device *vsen =
>>>>>>>                  container_of(sd, struct vimc_sen_device, sd);
>>>>>>>
>>>>>>>      v4l2_ctrl_handler_free(&vsen->hdl);
>>>>>>>      tpg_free(&vsen->tpg);
>>>>>>>      kfree(vsen);
>>>>>>> }
>>>>>>>
>>>>>>> And then you can see that `vsen` has been freed. Back to
>>>>>>> vimc_ent_sd_unregister(), after v4l2_device_unregister_subdev(), the
>>>>>>> function will call vimc_pads_cleanup(). This is basically a
>>>>>>> kfree(ved->pads), but `ved` has just been freed at
>>>>>>> v4l2_subdev_release(), producing a memory fault.
>>>>>>>
>>>>>>> To fix that, we found two options:
>>>>>>>
>>>>>>> - place the kfree(ved->pads) inside the release callback of each
>>>>>>> subdevice and removing vimc_pads_cleanup() from
>>>>>>> vimc_ent_sd_unregister()
>>>>>>> - use a auxiliary variable to hold the address of the pads, for instance:
>>>>>>>
>>>>>>> void vimc_ent_sd_unregister(...)
>>>>>>> {
>>>>>>>       struct media_pad *pads = ved->pads;
>>>>>>>       ...
>>>>>>>       vimc_pads_cleanup(pads);
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> I fixed a problem in the thirds patch. vimc-capture uses the first
>>>>>> approach - placing the kfree(ved->pads) inside the release callback.
>>>>>>
>>>>>> I am debugging another such problem in unbind path while streaming.
>>>>>> I am working on v2 and I will look for the rmmod problem and fix it.
>>>>>>
>>>>>> thanks again for testing and finding the root cause.
>>>>>> -- Shuah
>>>>>
>>>>> Hi Andre,
>>>>>
>>>>> Here is what's happening.
>>>>>
>>>>> Before this change, you can't really do rmmod vimc, because vimc is in
>>>>> use by other component drivers. With the collapse, now you can actually
>>>>> do rmmod on vimc and this problem in vimc_ent_sd_unregister() that frees
>>>>> pads first and the does v4l2_device_unregister_subdev().
>>>>>
>>>>> I fixed this in the 3/3 patch. I can reproduce the problem with patches 1 and 2, and patch 3 fixes it.
>>>>>
>>>>> Did you test with the third patch in this series?
>>>>
>>>> yes, we tested with 3/3, but the new problem now is when doing the following
>>>> in this order:
>>>>
>>>>      v4l2_device_unregister_subdev(sd);
>>>>      vimc_pads_cleanup(ved->pads);
>>>>
>>>>
>>>> v4l2_device_unregister_subdev() calls the release function of the subdevice that
>>>> frees the ved object, so ved->pads is not valid anymore. That is why André suggested
>>>> a temporary variable to hold ved->pads and to be able to free it later:
>>>>
>>>>      struct media_pad *pads = ved->pads;
>>>>
>>>>      v4l2_device_unregister_subdev(sd);
>>>>      vimc_pads_cleanup(pads); // So we don't use the ved object here anymore.
>>>
>>> Can't you simply call vimc_pads_cleanup() in the release function of the
>>> subdevice before freeing the ved object ?
>>
>> Yes we can, that is the other option Andre suggested.
>> The  advantage of doing it inside vimc_ent_sd_unregister() is that
>> who allocated the memory in the first place was vimc_ent_sd_register(), and also,
>> this is a common code to all subdevs, so letting it in vimc_ent_sd_unregister()
>> will make sure no subdevs ever forget to free this memory.
>> But saving the pointer to a variable to free it later is not that pretty
>> either.
> 
> The release function of a subdevice is meant to be used as a destructor,
> it should free all the resources allocated for the subdevice. Your
> subdevices (such as vimc_sca_device) are essentially objects derived
> from a common vimc subdevice object (vimc_ent_device), I thus suggest
> providing four functions to deal with vimc_ent_device:
> 
> - vimc_ent_sd_init() to initialise the base vimc_ent_device object (most
>    of the code is currently in vimc_ent_sd_register()), to be called by
>    the init function of each subdevice type (the bind functions in the
>    current code)
> 
> - vimc_ent_sd_register() to only register the vimc_ent_device once it
>    has been fully initialised
> 
> - vimc_end_sd_unregister() to unregister the vimc_ent_device
> 
> - vimc_end_sd_cleanup() to free the resources allocated by
>    vimc_ent_sd_init() (including calling vimc_pads_cleanup()), to be
>    called by the subdevice release handler
> 

We kind of mostly there. We have release handlers free'ing resources
with an exception of pads. I changed the release handlers to release
pads and with a comment where pads_init is called to say that pads
should be released from the release handler.

vimc-capture does this already. I changed the rest. rmmod works with
no issues now. rmmod when streaming comes back with in use.

There is one outstanding issue remaining in unbind path. I will fix
that in a separate patch. I decided to collapse the first two patches
into one and added common defines (vimc_device and vimc_ent_config) to
vimc-common.h based on our discussion.

I will send the patch series out soon.

thanks,
-- Shuah



thanks,
-- Shuah