From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E94D8004F for ; Tue, 19 Mar 2024 13:47:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710856036; cv=none; b=ZiJe/nEoeC6dOH/VI0Hxx9cLoD+4pEURDdFPyj1khTo+grcg6b0ZYOT+OIxP0bBd/vkK7SytfaXBFpXMfbMe0h9vbll1tT/Mxfjf3G0n68XhWw5Tfi1zFkTrPVM44w8kK2wMl8mh//HguPnAqDRwyCL1XFNFWLiTI/ZIubv6DeY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710856036; c=relaxed/simple; bh=U81OeYVZBANqfIy3A8NTA6kN2ZNwLOwqkzstbbsI+mY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZKoPW1wcUxzm5g5o+Z/kSQglrw4jJ83uGQc4pCCoLFWx0sepj3tZUpKFUPWrwooYQRrJFXyvfHaEaEFRVWLwfvp3PuS1X3rRJRQ3/yl/GQPyVmXiFXgiw0Z8PG7Mn8wnQreOfIEsv2X/HldL5bYaoyj8T7kSlYA/XLD/FbvgrM8= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Th3vDRgi; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Th3vDRgi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710856035; x=1742392035; h=date:from:to:cc:subject:message-id:references: mime-version:in-reply-to; bh=U81OeYVZBANqfIy3A8NTA6kN2ZNwLOwqkzstbbsI+mY=; b=Th3vDRgioSomwBMd0kanwc+KgIr1lMKhT3XstY8mmZygHCxfUvy5zKJ5 cUE3y7fjGvmTjA0qoKxTMR/lY0BqmSs8SaouxEioDkvTUv4QDGnm/7+fd qlci4crUHMwcOgyw1CCCJ52gwhr4z+TUTdK30dw3wT/RXpH9ntda/Bhh1 brcu7ZQRXvN4fEe8v0gZfHa69bSjtdNZAfymlOmimxRvnMwgYvLRdveTS QOs1LNJLt9aEKhySAZ7NNDE+tNvr4fkNYjh7P9tTLPcbWaeTwB+9cEevB /rkyfOyNqXeDu/O+1iABGd1VBosOO3N2HhBlZZM+gbh341X2uX371pgWR A==; X-IronPort-AV: E=McAfee;i="6600,9927,11017"; a="23184180" X-IronPort-AV: E=Sophos;i="6.07,137,1708416000"; d="scan'208";a="23184180" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2024 06:47:14 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,137,1708416000"; d="scan'208";a="44767911" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orviesa002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2024 06:47:11 -0700 Received: from kekkonen.localdomain (localhost [127.0.0.1]) by kekkonen.fi.intel.com (Postfix) with SMTP id CE62C11F853; Tue, 19 Mar 2024 15:47:07 +0200 (EET) Date: Tue, 19 Mar 2024 13:47:07 +0000 From: Sakari Ailus To: Julien Massot Cc: linux-media@vger.kernel.org, Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl, Andrey Konovalov , Jacopo Mondi , Dmitry Perchanov , "Ng, Khai Wen" , Alain Volmat Subject: Re: [PATCH v8 09/38] media: Documentation: v4l: Document internal source pads Message-ID: References: <20240313072516.241106-1-sakari.ailus@linux.intel.com> <20240313072516.241106-10-sakari.ailus@linux.intel.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Hi Julien, On Fri, Mar 15, 2024 at 04:32:59PM +0100, Julien Massot wrote: > > > On 3/13/24 08:24, Sakari Ailus wrote: > > Document internal source pads, pads that have both SINK and INTERNAL flags > > set. Use the IMX219 camera sensor as an example. > > > > Signed-off-by: Sakari Ailus > > --- > > .../userspace-api/media/v4l/dev-subdev.rst | 145 ++++++++++++++++++ > > 1 file changed, 145 insertions(+) > > > > diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst > > index a387e8a15b8d..1808f40f63e3 100644 > > --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst > > +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst > > @@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the > > sub-device and a (pad, stream) pair. For sub-devices that do not support > > multiplexed streams the 'stream' field is always 0. > > +.. _v4l2-subdev-internal-source-pads: > > + > > +Internal source pads and routing > > +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > + > > +Cases where a single sub-device source pad is traversed by multiple streams, one > > +or more of which originate from within the sub-device itself, are special as > > +there is no external sink pad for such routes. In those cases, the sources of > > +the internally generated streams are represented by internal source pads, which > > +are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL ` > > +pad flag set. > > + > > +Internal pads have all the properties of an external pad, including formats and > > +selections. The format in this case is the source format of the stream. An > > +internal pad always has a single stream only (0). > > + > > +Routes from an internal source pad to an external source pad are typically not > > +modifiable but they can be activated and deactivated using the > > +:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE ` flag, depending > > +on driver capabilities. > > + > > Interaction between routes, streams, formats and selections > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > @@ -668,3 +689,127 @@ To configure this pipeline, the userspace must take the following steps: > > the configurations along the stream towards the receiver, using > > :ref:`VIDIOC_SUBDEV_S_FMT ` ioctls to configure each > > stream endpoint in each sub-device. > > + > > +Internal pads setup example > > +^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > + > > +A simple example of a multiplexed stream setup might be as follows: > > + > > +- An IMX219 camera sensor source sub-device, with one sink pad (0), one source pad > > + (1), an internal sink pad (2) that represents the source of embedded > The pixel pad is an internal pad as well ? How about: - An IMX219 camera sensor source sub-device, with one internal sink pad (0) for image data, one source pad (1), an internal sink pad (2) that represents the source of embedded > > > + data. There are two routes, one from the sink pad to the source, and another > > + from the internal sink pad to the source pad. Both streams are always active, > > + i.e. there is no need to separately enable the embedded data stream. The > > + sensor uses the CSI-2 bus. > > + > > +- A CSI-2 receiver in the SoC (Receiver). The receiver has a single sink pad > > + (pad 0), connected to the bridge, and two source pads (pads 1-2), going to the > > + DMA engine. The receiver demultiplexes the incoming streams to the source > > + pads. > > + > > +- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is > > + connected to a single source pad in the receiver. > > + > > +The sensor, the bridge and the receiver are modeled as V4L2 sub-devices, > > +exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are > > +modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes. > > + > > +To configure this pipeline, the userspace must take the following steps: > > + > > +1) Set up media links between entities: connect the sensors to the bridge, > > + bridge to the receiver, and the receiver to the DMA engines. This step does > > + not differ from normal non-multiplexed media controller setup. > > + > > +2) Configure routing > > + > > +.. flat-table:: Camera sensor. There are no configurable routes. > > + :header-rows: 1 > > + > > + * - Sink Pad/Stream > > + - Source Pad/Stream > > + - Routing Flags > > + - Comments > > + * - 0/0 > > + - 1/0 > - 1/0 > - 0/0 > > + - V4L2_SUBDEV_ROUTE_FL_ACTIVE > > + - Pixel data stream from the sink pad > > + * - 2/0 > > + - 1/1 > - 0/1 > > + - V4L2_SUBDEV_ROUTE_FL_ACTIVE > > + - Metadata stream from the internal sink pad > > In latest patch "[PATCH v6 05/15] media: i2c: imx219: Add embedded data > support" > we have: > - 0 -> source pad > - 1 -> pixel/image > - 2 -> EDATA > > This is also what you did for ov2740. Yes, these were leftovers from the CCS example. I've fixed them. > > With that fixed: > Reviewed-by Julien Massot Thank you! -- Regards, Sakari Ailus