Linux-Renesas-SoC Archive on lore.kernel.org
 help / color / Atom feed
From: Sakari Ailus <sakari.ailus@linux.intel.com>
To: Jacopo Mondi <jacopo@jmondi.org>
Cc: "Niklas Söderlund" <niklas.soderlund+renesas@ragnatech.se>,
	"Laurent Pinchart" <laurent.pinchart@ideasonboard.com>,
	"Benoit Parrot" <bparrot@ti.com>,
	linux-media@vger.kernel.org, linux-renesas-soc@vger.kernel.org,
	"Michal Simek" <michal.simek@xilinx.com>
Subject: Re: [PATCH v2 16/30] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations
Date: Fri, 22 Feb 2019 13:04:29 +0200
Message-ID: <20190222110429.ybmqdwba5rszntb7@paasikivi.fi.intel.com> (raw)
In-Reply-To: <20190222084019.62atdkk6qipnugvf@uno.localdomain>

Hi Jacopo,

On Fri, Feb 22, 2019 at 09:40:19AM +0100, Jacopo Mondi wrote:
> Hi Sakari,
> 
> On Fri, Feb 22, 2019 at 12:31:32AM +0200, Sakari Ailus wrote:
> > Hi Jacopo,
> >
> > On Thu, Feb 21, 2019 at 03:39:40PM +0100, Jacopo Mondi wrote:
> > > Hi Sakari,
> > >    one quick question
> > >
> > > On Fri, Nov 02, 2018 at 12:31:30AM +0100, Niklas Söderlund wrote:
> > > > From: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > >
> > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > Signed-off-by: Michal Simek <michal.simek@xilinx.com>
> > > >
> > > > - Add sink and source streams for multiplexed links
> > > > - Copy the argument back in case of an error. This is needed to let the
> > > >   caller know the number of routes.
> > > >
> > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > Reviewed-by: Niklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
> > > > ---
> > > >  drivers/media/v4l2-core/v4l2-ioctl.c  | 20 +++++++++++++-
> > > >  drivers/media/v4l2-core/v4l2-subdev.c | 28 +++++++++++++++++++
> > > >  include/media/v4l2-subdev.h           |  7 +++++
> > > >  include/uapi/linux/v4l2-subdev.h      | 40 +++++++++++++++++++++++++++
> > > >  4 files changed, 94 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > index 7de041bae84fb2f2..40406acb51ec0906 100644
> > > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > > > @@ -19,6 +19,7 @@
> > > >  #include <linux/kernel.h>
> > > >  #include <linux/version.h>
> > > >
> > > > +#include <linux/v4l2-subdev.h>
> > > >  #include <linux/videodev2.h>
> > > >
> > > >  #include <media/v4l2-common.h>
> > > > @@ -2924,6 +2925,23 @@ static int check_array_args(unsigned int cmd, void *parg, size_t *array_size,
> > > >  		}
> > > >  		break;
> > > >  	}
> > > > +
> > > > +	case VIDIOC_SUBDEV_G_ROUTING:
> > > > +	case VIDIOC_SUBDEV_S_ROUTING: {
> > > > +		struct v4l2_subdev_routing *route = parg;
> > > > +
> > > > +		if (route->num_routes > 0) {
> > > > +			if (route->num_routes > 256)
> > > > +				return -EINVAL;
> > > > +
> > > > +			*user_ptr = (void __user *)route->routes;
> > > > +			*kernel_ptr = (void *)&route->routes;
> > > > +			*array_size = sizeof(struct v4l2_subdev_route)
> > > > +				    * route->num_routes;
> > > > +			ret = 1;
> > > > +		}
> > > > +		break;
> > > > +	}
> > > >  	}
> > > >
> > > >  	return ret;
> > > > @@ -3033,7 +3051,7 @@ video_usercopy(struct file *file, unsigned int cmd, unsigned long arg,
> > > >  	 * Some ioctls can return an error, but still have valid
> > > >  	 * results that must be returned.
> > > >  	 */
> > > > -	if (err < 0 && !always_copy)
> > > > +	if (err < 0 && !always_copy && cmd != VIDIOC_SUBDEV_G_ROUTING)
> > > >  		goto out;
> > > >
> > > >  out_array_args:
> > > > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c
> > > > index 792f41dffe2329b9..1d3b37cf548fa533 100644
> > > > --- a/drivers/media/v4l2-core/v4l2-subdev.c
> > > > +++ b/drivers/media/v4l2-core/v4l2-subdev.c
> > > > @@ -516,7 +516,35 @@ static long subdev_do_ioctl(struct file *file, unsigned int cmd, void *arg)
> > > >
> > > >  	case VIDIOC_SUBDEV_QUERYSTD:
> > > >  		return v4l2_subdev_call(sd, video, querystd, arg);
> > > > +
> > > > +	case VIDIOC_SUBDEV_G_ROUTING:
> > > > +		return v4l2_subdev_call(sd, pad, get_routing, arg);
> > > > +
> > > > +	case VIDIOC_SUBDEV_S_ROUTING: {
> > > > +		struct v4l2_subdev_routing *route = arg;
> > > > +		unsigned int i;
> > > > +
> > > > +		if (route->num_routes > sd->entity.num_pads)
> > > > +			return -EINVAL;
> > >
> > > Can't the number of routes exceeds the total number of pad?
> > >
> > > To make an example, a subdevice with 2 sink pads, and 1 multiplexed
> > > source pad, with 2 streams would expose the following routing table,
> > > right?
> > >
> > > pad #0 = sink, 1 stream
> > > pad #1 = sink, 1 stream
> > > pad #2 = source, 2 streams
> > >
> > > Routing table:
> > > 0/0 -> 2/0
> > > 0/0 -> 2/1
> > > 1/0 -> 2/0
> > > 1/0 -> 2/1
> > >
> > > In general, the number of accepted routes should depend on the number
> > > of streams, not pads, and that's better handled by drivers, am I
> > > wrong?
> >
> > Good point. Is the above configuration meaningful? I.e. you have a mux
> > device that does some processing as well by combining the streams?
> >
> > It's far-fetched but at the moment the API does not really bend for that.
> > It still might in the future.
> 
> Well, how would you expect a routing table for a device that
> supports multiplexing using Virtual Channels to look like?
> 
> As far as I got it, even with a single sink and a single source, it
> would be:
> 
> [SINK/SINK_STREAM -> SOURCE/SOURCE_STREAM]
> 0/0 -> 1/0
> 0/0 -> 1/1
> 0/0 -> 1/2
> 0/0 -> 1/3
> 
> On the previous example, I thought about GMSL-like devices, that can
> output the video streams received from different remotes in a
> separate virtual channel, at the same time.
> 
> A possible routing table in that case would be like:
> 
> Pads 0, 1, 2, 3 = SINKS
> Pad 4 = SOURCE with 4 streams (1 for each VC)
> 
> 0/0 -> 4/0
> 0/0 -> 4/1
> 0/0 -> 4/2
> 0/0 -> 4/3
> 1/0 -> 4/0
> 1/0 -> 4/1
> 1/0 -> 4/2
> 1/0 -> 4/3
> 2/0 -> 4/0
> 2/0 -> 4/1
> 2/0 -> 4/2
> 2/0 -> 4/3
> 3/0 -> 4/0
> 3/0 -> 4/1
> 3/0 -> 4/2
> 3/0 -> 4/3

If more than one pad can handle multiplexed streams, then you may end up in
a situation like that. Indeed.

> 
> With only one route per virtual channel active at a time.
> 
> Does this match your idea?
> 
> >
> > I guess we could just remove the check, and let drivers handle it.
> >
> 
> Yup, I agree!
> 
> > >
> > > Thanks
> > >   j
> > >
> > > > +
> > > > +		for (i = 0; i < route->num_routes; ++i) {
> > > > +			unsigned int sink = route->routes[i].sink_pad;
> > > > +			unsigned int source = route->routes[i].source_pad;
> > > > +			struct media_pad *pads = sd->entity.pads;
> > > > +
> > > > +			if (sink >= sd->entity.num_pads ||
> > > > +			    source >= sd->entity.num_pads)
> > > > +				return -EINVAL;
> > > > +
> > > > +			if (!(pads[sink].flags & MEDIA_PAD_FL_SINK) ||
> > > > +			    !(pads[source].flags & MEDIA_PAD_FL_SOURCE))
> > > > +				return -EINVAL;
> > > > +		}
> > > > +
> > > > +		return v4l2_subdev_call(sd, pad, set_routing, route);
> > > > +	}
> > > >  #endif
> > > > +
> > > >  	default:
> > > >  		return v4l2_subdev_call(sd, core, ioctl, cmd, arg);
> > > >  	}
> > > > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h
> > > > index 9102d6ca566e01f2..5acaeeb9b3cacefa 100644
> > > > --- a/include/media/v4l2-subdev.h
> > > > +++ b/include/media/v4l2-subdev.h
> > > > @@ -679,6 +679,9 @@ struct v4l2_subdev_pad_config {
> > > >   *
> > > >   * @set_frame_desc: set the low level media bus frame parameters, @fd array
> > > >   *                  may be adjusted by the subdev driver to device capabilities.
> > > > + *
> > > > + * @get_routing: callback for VIDIOC_SUBDEV_G_ROUTING IOCTL handler.
> > > > + * @set_routing: callback for VIDIOC_SUBDEV_S_ROUTING IOCTL handler.
> > > >   */
> > > >  struct v4l2_subdev_pad_ops {
> > > >  	int (*init_cfg)(struct v4l2_subdev *sd,
> > > > @@ -719,6 +722,10 @@ struct v4l2_subdev_pad_ops {
> > > >  			      struct v4l2_mbus_frame_desc *fd);
> > > >  	int (*set_frame_desc)(struct v4l2_subdev *sd, unsigned int pad,
> > > >  			      struct v4l2_mbus_frame_desc *fd);
> > > > +	int (*get_routing)(struct v4l2_subdev *sd,
> > > > +			   struct v4l2_subdev_routing *route);
> > > > +	int (*set_routing)(struct v4l2_subdev *sd,
> > > > +			   struct v4l2_subdev_routing *route);
> > > >  };
> > > >
> > > >  /**
> > > > diff --git a/include/uapi/linux/v4l2-subdev.h b/include/uapi/linux/v4l2-subdev.h
> > > > index 03970ce3074193e6..af069bfb10ca23a5 100644
> > > > --- a/include/uapi/linux/v4l2-subdev.h
> > > > +++ b/include/uapi/linux/v4l2-subdev.h
> > > > @@ -155,6 +155,44 @@ struct v4l2_subdev_selection {
> > > >  	__u32 reserved[8];
> > > >  };
> > > >
> > > > +#define V4L2_SUBDEV_ROUTE_FL_ACTIVE	(1 << 0)
> > > > +#define V4L2_SUBDEV_ROUTE_FL_IMMUTABLE	(1 << 1)
> > > > +
> > > > +/**
> > > > + * struct v4l2_subdev_route - A signal route inside a subdev
> > > > + * @sink_pad: the sink pad
> > > > + * @sink_stream: the sink stream
> > > > + * @source_pad: the source pad
> > > > + * @source_stream: the source stream
> > > > + * @flags: route flags:
> > > > + *
> > > > + *	V4L2_SUBDEV_ROUTE_FL_ACTIVE: Is the stream in use or not? An
> > > > + *	active stream will start when streaming is enabled on a video
> > > > + *	node. Set by the user.
> > > > + *
> > > > + *	V4L2_SUBDEV_ROUTE_FL_IMMUTABLE: Is the stream immutable, i.e.
> > > > + *	can it be activated and inactivated? Set by the driver.
> > > > + */
> > > > +struct v4l2_subdev_route {
> > > > +	__u32 sink_pad;
> > > > +	__u32 sink_stream;
> > > > +	__u32 source_pad;
> > > > +	__u32 source_stream;
> > > > +	__u32 flags;
> > > > +	__u32 reserved[5];
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct v4l2_subdev_routing - Routing information
> > > > + * @routes: the routes array
> > > > + * @num_routes: the total number of routes in the routes array
> > > > + */
> > > > +struct v4l2_subdev_routing {
> > > > +	struct v4l2_subdev_route *routes;
> >
> > This should be just __u64, to avoid writing compat code. The patch was
> > written before we started doing that. :-) Please see e.g. the media device
> > topology IOCTL.
> >
> 
> Thanks, I had a look at the MEDIA_ ioctls yesterday, G_TOPOLOGY in
> particular, which uses several pointers to arrays.
> 
> Unfortunately, I didn't come up with anything better than using a
> translation structure, from the IOCTL layer to the subdevice
> operations layer:
> https://paste.debian.net/hidden/b192969d/
> (sharing a link for early comments, I can send v3 and you can comment
> there directly if you prefer to :)

Hmm. That is a downside indeed. It's still a lesser problem than the compat
code in general --- which has been a source for bugs as well as nasty
security problems over time.

I think we need a naming scheme for such structs. How about just
calling that struct e.g. v4l2_subdev_krouting instead? It's simple, easy to
understand and it includes a suggestion which one is the kernel-only
variant.

You can btw. zero the struct memory by assigning { 0 } to it in
declaration. memset() in general is much more trouble. In this case you
could even do the assignments in delaration as well.

There is a check that requires that one pad is a source and another is a
sink; I think we could remove that check as well.

-- 
Regards,

Sakari Ailus
sakari.ailus@linux.intel.com

  reply index

Thread overview: 90+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-01 23:31 [PATCH v2 00/30] v4l: add support for multiplexed streams Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 01/30] media: entity: Use pad as a starting point for graph walk Niklas Söderlund
2019-01-15 21:43   ` Laurent Pinchart
2018-11-01 23:31 ` [PATCH v2 02/30] media: entity: Use pads instead of entities in the media graph walk stack Niklas Söderlund
2019-01-15 22:03   ` Laurent Pinchart
2019-01-15 22:13     ` Sakari Ailus
2019-01-15 22:07   ` Laurent Pinchart
2018-11-01 23:31 ` [PATCH v2 03/30] media: entity: Walk the graph based on pads Niklas Söderlund
2019-01-15 22:21   ` Laurent Pinchart
     [not found]     ` <20190115223406.mxgzl36cp54gb7nv@kekkonen.localdomain>
2019-01-15 23:28       ` Laurent Pinchart
2019-01-22 14:50         ` Sakari Ailus
2019-02-14 15:15   ` Jacopo Mondi
2018-11-01 23:31 ` [PATCH v2 04/30] v4l: mc: Start walk from a specific pad in use count calculation Niklas Söderlund
2019-01-15 22:24   ` Laurent Pinchart
2019-01-15 22:36     ` Sakari Ailus
2018-11-01 23:31 ` [PATCH v2 05/30] media: entity: Move the pipeline from entity to pads Niklas Söderlund
2019-01-15 22:38   ` Laurent Pinchart
2019-01-15 22:48     ` Sakari Ailus
2019-02-14 15:53   ` Jacopo Mondi
2018-11-01 23:31 ` [PATCH v2 06/30] media: entity: Use pad as the starting point for a pipeline Niklas Söderlund
2019-01-15 22:54   ` Laurent Pinchart
2019-01-22 15:31     ` Sakari Ailus
2019-01-22 15:37       ` Laurent Pinchart
2019-01-22 16:16         ` Sakari Ailus
2018-11-01 23:31 ` [PATCH v2 07/30] media: entity: Add has_route entity operation Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 08/30] media: entity: Add media_has_route() function Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 09/30] media: entity: Swap pads if route is checked from source to sink Niklas Söderlund
2019-01-15 22:57   ` Laurent Pinchart
2019-01-22 15:15     ` Sakari Ailus
2019-01-22 15:20       ` Laurent Pinchart
2019-02-18  9:21         ` Jacopo Mondi
2019-02-22 12:18           ` Laurent Pinchart
2019-03-04 12:35             ` Jacopo Mondi
2019-03-05 20:04               ` Laurent Pinchart
2019-03-06  8:29                 ` Jacopo Mondi
2018-11-01 23:31 ` [PATCH v2 10/30] media: entity: Use routing information during graph traversal Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 11/30] media: entity: Skip link validation for pads to which there is no route to Niklas Söderlund
2019-01-15 23:13   ` Laurent Pinchart
2018-11-01 23:31 ` [PATCH v2 12/30] media: entity: Add an iterator helper for connected pads Niklas Söderlund
2019-01-15 23:24   ` Laurent Pinchart
2019-01-22 15:36     ` Sakari Ailus
2019-01-22 15:38       ` Laurent Pinchart
2019-01-22 16:21         ` Sakari Ailus
2018-11-01 23:31 ` [PATCH v2 13/30] media: entity: Add only connected pads to the pipeline Niklas Söderlund
2019-01-15 23:33   ` Laurent Pinchart
2018-11-01 23:31 ` [PATCH v2 14/30] media: entity: Add debug information in graph walk route check Niklas Söderlund
2019-01-15 23:35   ` Laurent Pinchart
2019-01-22 15:38     ` Sakari Ailus
2018-11-01 23:31 ` [PATCH v2 15/30] media: entity: Look for indirect routes Niklas Söderlund
2019-01-15 23:41   ` Laurent Pinchart
2019-01-22 15:56     ` Sakari Ailus
2018-11-01 23:31 ` [PATCH v2 16/30] v4l: subdev: Add [GS]_ROUTING subdev ioctls and operations Niklas Söderlund
2019-01-15 23:51   ` Laurent Pinchart
2019-01-22 16:14     ` Sakari Ailus
2019-01-22 17:00       ` Laurent Pinchart
2019-02-21 14:59     ` Jacopo Mondi
2019-02-21 23:49       ` Sakari Ailus
2019-02-22  8:46         ` Jacopo Mondi
2019-02-21 14:39   ` Jacopo Mondi
2019-02-21 22:31     ` Sakari Ailus
2019-02-22  8:40       ` Jacopo Mondi
2019-02-22 11:04         ` Sakari Ailus [this message]
2019-02-22 11:17           ` Jacopo Mondi
2019-02-22 11:29             ` Sakari Ailus
2019-02-22 13:37               ` Ian Arkver
2019-02-22 13:50               ` Geert Uytterhoeven
2018-11-01 23:31 ` [PATCH v2 17/30] v4l: subdev: compat: Implement handling for VIDIOC_SUBDEV_[GS]_ROUTING Niklas Söderlund
2019-01-08 10:04   ` Geert Uytterhoeven
2019-01-15 23:53   ` Laurent Pinchart
2019-01-22 15:57     ` Sakari Ailus
2019-02-18 11:21     ` Jacopo Mondi
2019-02-21 23:50       ` Sakari Ailus
2018-11-01 23:31 ` [PATCH v2 18/30] v4l: subdev: Take routing information into account in link validation Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 19/30] v4l: subdev: Improve link format validation debug messages Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 20/30] v4l: mc: Add an S_ROUTING helper function for power state changes Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 21/30] v4l: Add bus type to frame descriptors Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 22/30] v4l: Add CSI-2 bus configuration " Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 23/30] v4l: Add stream to frame descriptor Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 24/30] adv748x: csi2: add translation from pixelcode to CSI-2 datatype Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 25/30] adv748x: csi2: only allow formats on sink pads Niklas Söderlund
2019-02-21 14:18   ` Jacopo Mondi
2018-11-01 23:31 ` [PATCH v2 26/30] adv748x: csi2: describe the multiplexed stream Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 27/30] adv748x: csi2: add internal routing configuration Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 28/30] adv748x: afe: add routing support Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 29/30] rcar-csi2: use frame description information to configure CSI-2 bus Niklas Söderlund
2018-11-01 23:31 ` [PATCH v2 30/30] rcar-csi2: expose the subdevice internal routing Niklas Söderlund
2018-11-14 13:10   ` Nikita Yushchenko
2018-11-14 19:45     ` Niklas Söderlund
2018-12-03 22:16 ` [PATCH v2 00/30] v4l: add support for multiplexed streams Sakari Ailus
2018-12-05 22:09   ` Niklas Söderlund

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190222110429.ybmqdwba5rszntb7@paasikivi.fi.intel.com \
    --to=sakari.ailus@linux.intel.com \
    --cc=bparrot@ti.com \
    --cc=jacopo@jmondi.org \
    --cc=laurent.pinchart@ideasonboard.com \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-renesas-soc@vger.kernel.org \
    --cc=michal.simek@xilinx.com \
    --cc=niklas.soderlund+renesas@ragnatech.se \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Renesas-SoC Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-renesas-soc/0 linux-renesas-soc/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-renesas-soc linux-renesas-soc/ https://lore.kernel.org/linux-renesas-soc \
		linux-renesas-soc@vger.kernel.org linux-renesas-soc@archiver.kernel.org
	public-inbox-index linux-renesas-soc


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-renesas-soc


AGPL code for this site: git clone https://public-inbox.org/ public-inbox