* [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 13:48 ` Hans Verkuil
2020-05-04 9:25 ` [PATCH v2 02/34] media: uapi: Add MEDIA_BUS_FMT_SENSOR_DATA media bus format Laurent Pinchart
` (33 subsequent siblings)
34 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Naushir Patuck <naush@raspberrypi.com>
Add V4L2_META_FMT_SENSOR_DATA format 4CC.
This new format will be used by the BCM2835 Unicam device to return
out camera sensor embedded data.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
---
Documentation/media/uapi/v4l/meta-formats.rst | 1 +
.../uapi/v4l/pixfmt-meta-sensor-data.rst | 32 +++++++++++++++++++
drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
include/uapi/linux/videodev2.h | 1 +
4 files changed, 35 insertions(+)
create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
diff --git a/Documentation/media/uapi/v4l/meta-formats.rst b/Documentation/media/uapi/v4l/meta-formats.rst
index 74c8659ee9d6..5474086ef6f0 100644
--- a/Documentation/media/uapi/v4l/meta-formats.rst
+++ b/Documentation/media/uapi/v4l/meta-formats.rst
@@ -21,6 +21,7 @@ These formats are used for the :ref:`metadata` interface only.
pixfmt-meta-d4xx
pixfmt-meta-intel-ipu3
+ pixfmt-meta-sensor-data
pixfmt-meta-uvc
pixfmt-meta-vsp1-hgo
pixfmt-meta-vsp1-hgt
diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
new file mode 100644
index 000000000000..4a67e204d08a
--- /dev/null
+++ b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
@@ -0,0 +1,32 @@
+.. Permission is granted to copy, distribute and/or modify this
+.. document under the terms of the GNU Free Documentation License,
+.. Version 1.1 or any later version published by the Free Software
+.. Foundation, with no Invariant Sections, no Front-Cover Texts
+.. and no Back-Cover Texts. A copy of the license is included at
+.. Documentation/media/uapi/fdl-appendix.rst.
+..
+.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
+
+.. _v4l2-meta-fmt-sensor-data:
+
+***********************************
+V4L2_META_FMT_SENSOR_DATA ('SENS')
+***********************************
+
+Sensor Ancillary Metadata
+
+Description
+===========
+
+This format describes ancillary data generated by a camera sensor and
+transmitted over a stream on the camera bus. Sensor vendors generally have their
+own custom format for this ancillary data. Some vendors follow a generic
+CSI-2/SMIA embedded data format as described in the `CSI-2 specification.
+<https://mipi.org/specifications/csi-2>`_
+
+The size of the embedded buffer is defined as a single line with a pixel width
+width specified in bytes. This is obtained by a call to the
+:c:type:`VIDIOC_SUBDEV_G_FMT` ioctl on the sensor subdevice where the ``pad``
+field in :c:type:`v4l2_subdev_format` is set to 1. Note that this size is fixed
+and cannot be modified with a call to :c:type:`VIDIOC_SUBDEV_S_FMT`.
+
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index b2ef8e60ea7d..faf5a0f5eb6b 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1346,6 +1346,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata"; break;
case V4L2_META_FMT_D4XX: descr = "Intel D4xx UVC Metadata"; break;
case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
+ case V4L2_META_FMT_SENSOR_DATA: descr = "Sensor Ancillary Metadata"; break;
default:
/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 9817b7e2c968..a96146223843 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -766,6 +766,7 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
#define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
#define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
+#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S') /* Sensor Ancillary metadata */
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type
2020-05-04 9:25 ` [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type Laurent Pinchart
@ 2020-05-04 13:48 ` Hans Verkuil
2020-05-04 14:39 ` Dave Stevenson
0 siblings, 1 reply; 111+ messages in thread
From: Hans Verkuil @ 2020-05-04 13:48 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
On 04/05/2020 11:25, Laurent Pinchart wrote:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Add V4L2_META_FMT_SENSOR_DATA format 4CC.
>
> This new format will be used by the BCM2835 Unicam device to return
> out camera sensor embedded data.
out -> our (I think)
>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> ---
> Documentation/media/uapi/v4l/meta-formats.rst | 1 +
> .../uapi/v4l/pixfmt-meta-sensor-data.rst | 32 +++++++++++++++++++
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> include/uapi/linux/videodev2.h | 1 +
> 4 files changed, 35 insertions(+)
> create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
>
> diff --git a/Documentation/media/uapi/v4l/meta-formats.rst b/Documentation/media/uapi/v4l/meta-formats.rst
> index 74c8659ee9d6..5474086ef6f0 100644
> --- a/Documentation/media/uapi/v4l/meta-formats.rst
> +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> @@ -21,6 +21,7 @@ These formats are used for the :ref:`metadata` interface only.
>
> pixfmt-meta-d4xx
> pixfmt-meta-intel-ipu3
> + pixfmt-meta-sensor-data
> pixfmt-meta-uvc
> pixfmt-meta-vsp1-hgo
> pixfmt-meta-vsp1-hgt
> diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> new file mode 100644
> index 000000000000..4a67e204d08a
> --- /dev/null
> +++ b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> @@ -0,0 +1,32 @@
> +.. Permission is granted to copy, distribute and/or modify this
> +.. document under the terms of the GNU Free Documentation License,
> +.. Version 1.1 or any later version published by the Free Software
> +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> +.. and no Back-Cover Texts. A copy of the license is included at
> +.. Documentation/media/uapi/fdl-appendix.rst.
> +..
> +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> +
> +.. _v4l2-meta-fmt-sensor-data:
> +
> +***********************************
> +V4L2_META_FMT_SENSOR_DATA ('SENS')
> +***********************************
> +
> +Sensor Ancillary Metadata
> +
> +Description
> +===========
> +
> +This format describes ancillary data generated by a camera sensor and
> +transmitted over a stream on the camera bus. Sensor vendors generally have their
'generally' can be dropped.
> +own custom format for this ancillary data. Some vendors follow a generic
> +CSI-2/SMIA embedded data format as described in the `CSI-2 specification.
> +<https://mipi.org/specifications/csi-2>`_
So what does this format represent? Sensor specific custom data where you need
sensor-specific NDA knowledge in userspace to parse it? Or SMIA formatted sensor
data?
A 4cc should map to a specific data format. I understand that this is not always
known, but at least the SMIA data format would need its own 4cc. I assume that the
sensor driver will at least know that it produces SMIA data, right?
"SENSOR_DATA" is also much too generic IMHO.
> +
> +The size of the embedded buffer is defined as a single line with a pixel width
> +width specified in bytes. This is obtained by a call to the
"pixel width width"? Typo?
> +:c:type:`VIDIOC_SUBDEV_G_FMT` ioctl on the sensor subdevice where the ``pad``
> +field in :c:type:`v4l2_subdev_format` is set to 1. Note that this size is fixed
Should this description be explicit about the pad number?
Regards,
Hans
> +and cannot be modified with a call to :c:type:`VIDIOC_SUBDEV_S_FMT`.
> +
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index b2ef8e60ea7d..faf5a0f5eb6b 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1346,6 +1346,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata"; break;
> case V4L2_META_FMT_D4XX: descr = "Intel D4xx UVC Metadata"; break;
> case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
> + case V4L2_META_FMT_SENSOR_DATA: descr = "Sensor Ancillary Metadata"; break;
>
> default:
> /* Compressed formats */
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 9817b7e2c968..a96146223843 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -766,6 +766,7 @@ struct v4l2_pix_format {
> #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> #define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> #define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
> +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S') /* Sensor Ancillary metadata */
>
> /* priv field value to indicates that subsequent fields are valid. */
> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
>
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type
2020-05-04 13:48 ` Hans Verkuil
@ 2020-05-04 14:39 ` Dave Stevenson
2020-05-04 15:32 ` Hans Verkuil
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-05-04 14:39 UTC (permalink / raw)
To: Hans Verkuil
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
Hi Hans
On Mon, 4 May 2020 at 14:48, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 04/05/2020 11:25, Laurent Pinchart wrote:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Add V4L2_META_FMT_SENSOR_DATA format 4CC.
> >
> > This new format will be used by the BCM2835 Unicam device to return
> > out camera sensor embedded data.
>
> out -> our (I think)
More like drop the word totally.
"used by BCM2835 Unicam device to return camera sensor embedded data"
The reference to Unicam probably ought to be dropped as other SoCs may
also be able to return embedded data.
> >
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > ---
> > Documentation/media/uapi/v4l/meta-formats.rst | 1 +
> > .../uapi/v4l/pixfmt-meta-sensor-data.rst | 32 +++++++++++++++++++
> > drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> > include/uapi/linux/videodev2.h | 1 +
> > 4 files changed, 35 insertions(+)
> > create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> >
> > diff --git a/Documentation/media/uapi/v4l/meta-formats.rst b/Documentation/media/uapi/v4l/meta-formats.rst
> > index 74c8659ee9d6..5474086ef6f0 100644
> > --- a/Documentation/media/uapi/v4l/meta-formats.rst
> > +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> > @@ -21,6 +21,7 @@ These formats are used for the :ref:`metadata` interface only.
> >
> > pixfmt-meta-d4xx
> > pixfmt-meta-intel-ipu3
> > + pixfmt-meta-sensor-data
> > pixfmt-meta-uvc
> > pixfmt-meta-vsp1-hgo
> > pixfmt-meta-vsp1-hgt
> > diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> > new file mode 100644
> > index 000000000000..4a67e204d08a
> > --- /dev/null
> > +++ b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> > @@ -0,0 +1,32 @@
> > +.. Permission is granted to copy, distribute and/or modify this
> > +.. document under the terms of the GNU Free Documentation License,
> > +.. Version 1.1 or any later version published by the Free Software
> > +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> > +.. and no Back-Cover Texts. A copy of the license is included at
> > +.. Documentation/media/uapi/fdl-appendix.rst.
> > +..
> > +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> > +
> > +.. _v4l2-meta-fmt-sensor-data:
> > +
> > +***********************************
> > +V4L2_META_FMT_SENSOR_DATA ('SENS')
> > +***********************************
> > +
> > +Sensor Ancillary Metadata
> > +
> > +Description
> > +===========
> > +
> > +This format describes ancillary data generated by a camera sensor and
> > +transmitted over a stream on the camera bus. Sensor vendors generally have their
>
> 'generally' can be dropped.
>
> > +own custom format for this ancillary data. Some vendors follow a generic
> > +CSI-2/SMIA embedded data format as described in the `CSI-2 specification.
> > +<https://mipi.org/specifications/csi-2>`_
>
> So what does this format represent? Sensor specific custom data where you need
> sensor-specific NDA knowledge in userspace to parse it? Or SMIA formatted sensor
> data?
>
> A 4cc should map to a specific data format. I understand that this is not always
> known, but at least the SMIA data format would need its own 4cc. I assume that the
> sensor driver will at least know that it produces SMIA data, right?
>
> "SENSOR_DATA" is also much too generic IMHO.
It's a tricky one to call as to either creating a vast number of
formats vs being too generic.
With V4L2 setting exposure as a number of lines, and analogue gain as
the precomputed the gain code, decoding the register set from embedded
data almost always requires sensor specific information.
SMIA defines a strict register set. Yes you could decode that using
the defined rules. However I'm not aware of any current sensors that
actually implement the SMIA spec - that pretty much died with Nokia.
Looking slightly more generically, there is the encoding of the
embedded data based on the Bayer format
Raw 10 puts a dummy byte in every 5th byte to "match" the Bayer
encoding of the least sig bits for 4 pixels being packed in the 5th
byte.
Raw 12 puts a dummy byte in every 3rd byte to "match" the Bayer raw 12 format.
Raw 14 I have no idea over with 3 pixels in 7 bytes packing.
(Raw 8 has no additional bytes, and I wouldn't expect raw 16 to have
any either).
This could be defined as a SMIA-style packing, and is very common.
They also tend to use the same 0x55 0x5a 0xa5 0xaa values for
signalling for which registers are used number is being presented.
So you could reasonably define a SENSOR_DATA_RAW8, SENSOR_DATA_RAW10,
SENSOR_DATA_RAW12, SENSOR_DATA_RAW14, and SENSOR_DATA_RAW16 formats,
but userland still needs to know in which registers the actual useful
values are.
You've also now mandated support of V4L2_EVENT_SOURCE_CHANGE events,
as changing the Bayer format on the video node will change the
metadata format on the other.
However unless you also know the exact sensor, you still don't know
which register(s) relates to exposure or analogue gain. To do that
would imply a unique 4CC in both mediabus and V4L2 domains for every
sensor that varies the register configuration (families of sensors may
put them in the same place if you're lucky). And you'll need
raw10/12/14 packing variants of each based on the formats supported by
the sensor.
So as I see it you have 3 levels:
1) Totally generic and let userspace have to worry about the problem.
(1 new format)
2) Semi generic raw8/10/12/14/16 packing for the data, but still
userspace needs to know sensor specifics to decode anything useful
from it (5 new formats)
3) Unique formats for every sensor, and for each bit depth that that
sensor can produce (I'd estimate around ~20 new formats for the
drivers in existing mainline kernel, and probably adding 1 or 2 per
new driver).
Personally I don't see the third option as being practical, and the
second doesn't really add anything that significant over the first as
you still need to know the register setup.
You can't even easily take the second option and add read-only
controls telling userspace the registers for exposure and gain (and
anything else?). You'd also need the info on how many bits of each
register(s) are used, for exposure are all bits denoting whole lines,
or and do you compute the analogue gain code (this can often be
defined via 4 co-efficients, but that's not guaranteed). Perhaps it
can be described in a dozen new read only controls if you're lucky,
but then there's bound to be one sensor that can't.
So where do you draw the line between too generic and totally
unscalable? Over to you.
Regards
Dave
> > +
> > +The size of the embedded buffer is defined as a single line with a pixel width
> > +width specified in bytes. This is obtained by a call to the
>
> "pixel width width"? Typo?
>
> > +:c:type:`VIDIOC_SUBDEV_G_FMT` ioctl on the sensor subdevice where the ``pad``
> > +field in :c:type:`v4l2_subdev_format` is set to 1. Note that this size is fixed
>
> Should this description be explicit about the pad number?
>
> Regards,
>
> Hans
>
> > +and cannot be modified with a call to :c:type:`VIDIOC_SUBDEV_S_FMT`.
> > +
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index b2ef8e60ea7d..faf5a0f5eb6b 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1346,6 +1346,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata"; break;
> > case V4L2_META_FMT_D4XX: descr = "Intel D4xx UVC Metadata"; break;
> > case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
> > + case V4L2_META_FMT_SENSOR_DATA: descr = "Sensor Ancillary Metadata"; break;
> >
> > default:
> > /* Compressed formats */
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 9817b7e2c968..a96146223843 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -766,6 +766,7 @@ struct v4l2_pix_format {
> > #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> > #define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> > #define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
> > +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S') /* Sensor Ancillary metadata */
> >
> > /* priv field value to indicates that subsequent fields are valid. */
> > #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
> >
>
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type
2020-05-04 14:39 ` Dave Stevenson
@ 2020-05-04 15:32 ` Hans Verkuil
2020-05-04 16:08 ` Laurent Pinchart
0 siblings, 1 reply; 111+ messages in thread
From: Hans Verkuil @ 2020-05-04 15:32 UTC (permalink / raw)
To: Dave Stevenson
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
On 04/05/2020 16:39, Dave Stevenson wrote:
> Hi Hans
>
> On Mon, 4 May 2020 at 14:48, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 04/05/2020 11:25, Laurent Pinchart wrote:
>>> From: Naushir Patuck <naush@raspberrypi.com>
>>>
>>> Add V4L2_META_FMT_SENSOR_DATA format 4CC.
>>>
>>> This new format will be used by the BCM2835 Unicam device to return
>>> out camera sensor embedded data.
>>
>> out -> our (I think)
>
> More like drop the word totally.
> "used by BCM2835 Unicam device to return camera sensor embedded data"
>
> The reference to Unicam probably ought to be dropped as other SoCs may
> also be able to return embedded data.
>
>>>
>>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
>>> ---
>>> Documentation/media/uapi/v4l/meta-formats.rst | 1 +
>>> .../uapi/v4l/pixfmt-meta-sensor-data.rst | 32 +++++++++++++++++++
>>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
>>> include/uapi/linux/videodev2.h | 1 +
>>> 4 files changed, 35 insertions(+)
>>> create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
>>>
>>> diff --git a/Documentation/media/uapi/v4l/meta-formats.rst b/Documentation/media/uapi/v4l/meta-formats.rst
>>> index 74c8659ee9d6..5474086ef6f0 100644
>>> --- a/Documentation/media/uapi/v4l/meta-formats.rst
>>> +++ b/Documentation/media/uapi/v4l/meta-formats.rst
>>> @@ -21,6 +21,7 @@ These formats are used for the :ref:`metadata` interface only.
>>>
>>> pixfmt-meta-d4xx
>>> pixfmt-meta-intel-ipu3
>>> + pixfmt-meta-sensor-data
>>> pixfmt-meta-uvc
>>> pixfmt-meta-vsp1-hgo
>>> pixfmt-meta-vsp1-hgt
>>> diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
>>> new file mode 100644
>>> index 000000000000..4a67e204d08a
>>> --- /dev/null
>>> +++ b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
>>> @@ -0,0 +1,32 @@
>>> +.. Permission is granted to copy, distribute and/or modify this
>>> +.. document under the terms of the GNU Free Documentation License,
>>> +.. Version 1.1 or any later version published by the Free Software
>>> +.. Foundation, with no Invariant Sections, no Front-Cover Texts
>>> +.. and no Back-Cover Texts. A copy of the license is included at
>>> +.. Documentation/media/uapi/fdl-appendix.rst.
>>> +..
>>> +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
>>> +
>>> +.. _v4l2-meta-fmt-sensor-data:
>>> +
>>> +***********************************
>>> +V4L2_META_FMT_SENSOR_DATA ('SENS')
>>> +***********************************
>>> +
>>> +Sensor Ancillary Metadata
>>> +
>>> +Description
>>> +===========
>>> +
>>> +This format describes ancillary data generated by a camera sensor and
>>> +transmitted over a stream on the camera bus. Sensor vendors generally have their
>>
>> 'generally' can be dropped.
>>
>>> +own custom format for this ancillary data. Some vendors follow a generic
>>> +CSI-2/SMIA embedded data format as described in the `CSI-2 specification.
>>> +<https://mipi.org/specifications/csi-2>`_
>>
>> So what does this format represent? Sensor specific custom data where you need
>> sensor-specific NDA knowledge in userspace to parse it? Or SMIA formatted sensor
>> data?
>>
>> A 4cc should map to a specific data format. I understand that this is not always
>> known, but at least the SMIA data format would need its own 4cc. I assume that the
>> sensor driver will at least know that it produces SMIA data, right?
>>
>> "SENSOR_DATA" is also much too generic IMHO.
>
> It's a tricky one to call as to either creating a vast number of
> formats vs being too generic.
>
> With V4L2 setting exposure as a number of lines, and analogue gain as
> the precomputed the gain code, decoding the register set from embedded
> data almost always requires sensor specific information.
>
> SMIA defines a strict register set. Yes you could decode that using
> the defined rules. However I'm not aware of any current sensors that
> actually implement the SMIA spec - that pretty much died with Nokia.
OK.
>
> Looking slightly more generically, there is the encoding of the
> embedded data based on the Bayer format
> Raw 10 puts a dummy byte in every 5th byte to "match" the Bayer
> encoding of the least sig bits for 4 pixels being packed in the 5th
> byte.
> Raw 12 puts a dummy byte in every 3rd byte to "match" the Bayer raw 12 format.
> Raw 14 I have no idea over with 3 pixels in 7 bytes packing.
> (Raw 8 has no additional bytes, and I wouldn't expect raw 16 to have
> any either).
> This could be defined as a SMIA-style packing, and is very common.
> They also tend to use the same 0x55 0x5a 0xa5 0xaa values for
> signalling for which registers are used number is being presented.
>
> So you could reasonably define a SENSOR_DATA_RAW8, SENSOR_DATA_RAW10,
> SENSOR_DATA_RAW12, SENSOR_DATA_RAW14, and SENSOR_DATA_RAW16 formats,
> but userland still needs to know in which registers the actual useful
> values are.
> You've also now mandated support of V4L2_EVENT_SOURCE_CHANGE events,
> as changing the Bayer format on the video node will change the
> metadata format on the other.
>
> However unless you also know the exact sensor, you still don't know
> which register(s) relates to exposure or analogue gain. To do that
> would imply a unique 4CC in both mediabus and V4L2 domains for every
> sensor that varies the register configuration (families of sensors may
> put them in the same place if you're lucky). And you'll need
> raw10/12/14 packing variants of each based on the formats supported by
> the sensor.
Out of curiosity: is the description of this metadata still something
that is typically under NDA? Or is this opening up more?
>
> So as I see it you have 3 levels:
> 1) Totally generic and let userspace have to worry about the problem.
> (1 new format)
> 2) Semi generic raw8/10/12/14/16 packing for the data, but still
> userspace needs to know sensor specifics to decode anything useful
> from it (5 new formats)
> 3) Unique formats for every sensor, and for each bit depth that that
> sensor can produce (I'd estimate around ~20 new formats for the
> drivers in existing mainline kernel, and probably adding 1 or 2 per
> new driver).
>
> Personally I don't see the third option as being practical, and the
> second doesn't really add anything that significant over the first as
> you still need to know the register setup.
>
> You can't even easily take the second option and add read-only
> controls telling userspace the registers for exposure and gain (and
> anything else?). You'd also need the info on how many bits of each
> register(s) are used, for exposure are all bits denoting whole lines,
> or and do you compute the analogue gain code (this can often be
> defined via 4 co-efficients, but that's not guaranteed). Perhaps it
> can be described in a dozen new read only controls if you're lucky,
> but then there's bound to be one sensor that can't.
>
>
> So where do you draw the line between too generic and totally
> unscalable? Over to you.
So a proper description of this format would be that it is custom sensor
specific data stored in the first line of the image sent by the sensor.
Since this is sensor specific, how does libcamera know which sensor it is?
It uses the sensor entity information for that?
Regards,
Hans
>
> Regards
> Dave
>
>>> +
>>> +The size of the embedded buffer is defined as a single line with a pixel width
>>> +width specified in bytes. This is obtained by a call to the
>>
>> "pixel width width"? Typo?
>>
>>> +:c:type:`VIDIOC_SUBDEV_G_FMT` ioctl on the sensor subdevice where the ``pad``
>>> +field in :c:type:`v4l2_subdev_format` is set to 1. Note that this size is fixed
>>
>> Should this description be explicit about the pad number?
>>
>> Regards,
>>
>> Hans
>>
>>> +and cannot be modified with a call to :c:type:`VIDIOC_SUBDEV_S_FMT`.
>>> +
>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index b2ef8e60ea7d..faf5a0f5eb6b 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -1346,6 +1346,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>> case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata"; break;
>>> case V4L2_META_FMT_D4XX: descr = "Intel D4xx UVC Metadata"; break;
>>> case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
>>> + case V4L2_META_FMT_SENSOR_DATA: descr = "Sensor Ancillary Metadata"; break;
>>>
>>> default:
>>> /* Compressed formats */
>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index 9817b7e2c968..a96146223843 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -766,6 +766,7 @@ struct v4l2_pix_format {
>>> #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
>>> #define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
>>> #define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
>>> +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S') /* Sensor Ancillary metadata */
>>>
>>> /* priv field value to indicates that subsequent fields are valid. */
>>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
>>>
>>
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type
2020-05-04 15:32 ` Hans Verkuil
@ 2020-05-04 16:08 ` Laurent Pinchart
2020-05-05 11:20 ` Dave Stevenson
0 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 16:08 UTC (permalink / raw)
To: Hans Verkuil
Cc: Dave Stevenson, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
Hi Hans,
On Mon, May 04, 2020 at 05:32:42PM +0200, Hans Verkuil wrote:
> On 04/05/2020 16:39, Dave Stevenson wrote:
> > On Mon, 4 May 2020 at 14:48, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> >> On 04/05/2020 11:25, Laurent Pinchart wrote:
> >>> From: Naushir Patuck <naush@raspberrypi.com>
> >>>
> >>> Add V4L2_META_FMT_SENSOR_DATA format 4CC.
> >>>
> >>> This new format will be used by the BCM2835 Unicam device to return
> >>> out camera sensor embedded data.
> >>
> >> out -> our (I think)
> >
> > More like drop the word totally.
> > "used by BCM2835 Unicam device to return camera sensor embedded data"
> >
> > The reference to Unicam probably ought to be dropped as other SoCs may
> > also be able to return embedded data.
> >
> >>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> >>> ---
> >>> Documentation/media/uapi/v4l/meta-formats.rst | 1 +
> >>> .../uapi/v4l/pixfmt-meta-sensor-data.rst | 32 +++++++++++++++++++
> >>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> >>> include/uapi/linux/videodev2.h | 1 +
> >>> 4 files changed, 35 insertions(+)
> >>> create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> >>>
> >>> diff --git a/Documentation/media/uapi/v4l/meta-formats.rst b/Documentation/media/uapi/v4l/meta-formats.rst
> >>> index 74c8659ee9d6..5474086ef6f0 100644
> >>> --- a/Documentation/media/uapi/v4l/meta-formats.rst
> >>> +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> >>> @@ -21,6 +21,7 @@ These formats are used for the :ref:`metadata` interface only.
> >>>
> >>> pixfmt-meta-d4xx
> >>> pixfmt-meta-intel-ipu3
> >>> + pixfmt-meta-sensor-data
> >>> pixfmt-meta-uvc
> >>> pixfmt-meta-vsp1-hgo
> >>> pixfmt-meta-vsp1-hgt
> >>> diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> >>> new file mode 100644
> >>> index 000000000000..4a67e204d08a
> >>> --- /dev/null
> >>> +++ b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> >>> @@ -0,0 +1,32 @@
> >>> +.. Permission is granted to copy, distribute and/or modify this
> >>> +.. document under the terms of the GNU Free Documentation License,
> >>> +.. Version 1.1 or any later version published by the Free Software
> >>> +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> >>> +.. and no Back-Cover Texts. A copy of the license is included at
> >>> +.. Documentation/media/uapi/fdl-appendix.rst.
> >>> +..
> >>> +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> >>> +
> >>> +.. _v4l2-meta-fmt-sensor-data:
> >>> +
> >>> +***********************************
> >>> +V4L2_META_FMT_SENSOR_DATA ('SENS')
> >>> +***********************************
> >>> +
> >>> +Sensor Ancillary Metadata
> >>> +
> >>> +Description
> >>> +===========
> >>> +
> >>> +This format describes ancillary data generated by a camera sensor and
> >>> +transmitted over a stream on the camera bus. Sensor vendors generally have their
> >>
> >> 'generally' can be dropped.
> >>
> >>> +own custom format for this ancillary data. Some vendors follow a generic
> >>> +CSI-2/SMIA embedded data format as described in the `CSI-2 specification.
> >>> +<https://mipi.org/specifications/csi-2>`_
> >>
> >> So what does this format represent? Sensor specific custom data where you need
> >> sensor-specific NDA knowledge in userspace to parse it? Or SMIA formatted sensor
> >> data?
> >>
> >> A 4cc should map to a specific data format. I understand that this is not always
> >> known, but at least the SMIA data format would need its own 4cc. I assume that the
> >> sensor driver will at least know that it produces SMIA data, right?
> >>
> >> "SENSOR_DATA" is also much too generic IMHO.
> >
> > It's a tricky one to call as to either creating a vast number of
> > formats vs being too generic.
> >
> > With V4L2 setting exposure as a number of lines, and analogue gain as
> > the precomputed the gain code, decoding the register set from embedded
> > data almost always requires sensor specific information.
> >
> > SMIA defines a strict register set. Yes you could decode that using
> > the defined rules. However I'm not aware of any current sensors that
> > actually implement the SMIA spec - that pretty much died with Nokia.
>
> OK.
Two points worth noting here:
- SMIA and SMIA++ indeed died with Nokia, but quite a few sensors are
still at least partly compatible with the specs, as sensor vendors
have a tendency to start development of new sensors based on older
hardware implementations. I don't know to what extent this applies to
metadata, or to recent sensors.
- MIPI has released the Camera Command Set (CCS) specification
(https://www.mipi.org/specifications/camera-command-set), which is
similar to SMIA++ for the command set part. Register offsets are
different, but the sensor model and exposed data is similar. The
specification standardizes the format of embedded data, and in
practice the data we're interested in is expressed as a set of
tag/value pairs. Tags are more or less register addresses, and I think
vendor-specific tags are allowed (as vendor-specific registers are
also allowed). The specification isn't widely implemented yet, but
some sensors that are not advertized as CCS-compliant actually follow
the specification to some extent (again with more or less variation
depending on the sensor model).
The CCS specification is public, you can check it out.
> > Looking slightly more generically, there is the encoding of the
> > embedded data based on the Bayer format
> > Raw 10 puts a dummy byte in every 5th byte to "match" the Bayer
> > encoding of the least sig bits for 4 pixels being packed in the 5th
> > byte.
> > Raw 12 puts a dummy byte in every 3rd byte to "match" the Bayer raw 12 format.
> > Raw 14 I have no idea over with 3 pixels in 7 bytes packing.
> > (Raw 8 has no additional bytes, and I wouldn't expect raw 16 to have
> > any either).
There are two options in RAW16, efficient packing with 2 bytes per
sample, or legacy packing with one byte per sample.
> > This could be defined as a SMIA-style packing, and is very common.
> > They also tend to use the same 0x55 0x5a 0xa5 0xaa values for
> > signalling for which registers are used number is being presented.
> >
> > So you could reasonably define a SENSOR_DATA_RAW8, SENSOR_DATA_RAW10,
> > SENSOR_DATA_RAW12, SENSOR_DATA_RAW14, and SENSOR_DATA_RAW16 formats,
> > but userland still needs to know in which registers the actual useful
> > values are.
> > You've also now mandated support of V4L2_EVENT_SOURCE_CHANGE events,
> > as changing the Bayer format on the video node will change the
> > metadata format on the other.
> >
> > However unless you also know the exact sensor, you still don't know
> > which register(s) relates to exposure or analogue gain. To do that
> > would imply a unique 4CC in both mediabus and V4L2 domains for every
> > sensor that varies the register configuration (families of sensors may
> > put them in the same place if you're lucky). And you'll need
> > raw10/12/14 packing variants of each based on the formats supported by
> > the sensor.
>
> Out of curiosity: is the description of this metadata still something
> that is typically under NDA? Or is this opening up more?
It varies between sensor vendors. For CCS-compliant sensors most of the
information is in the spec. The vendor-specific registers are usually
under NDA, but some vendors release datasheets, at least for some of
their products (for instance Aptina, now On Semi, has traditionally been
more open than OmniVision). I don't know if moving towards CCS
compliance will have the undesired side effect of leaving
vendor-specific registers not documented.
> > So as I see it you have 3 levels:
> > 1) Totally generic and let userspace have to worry about the problem.
> > (1 new format)
> > 2) Semi generic raw8/10/12/14/16 packing for the data, but still
> > userspace needs to know sensor specifics to decode anything useful
> > from it (5 new formats)
> > 3) Unique formats for every sensor, and for each bit depth that that
> > sensor can produce (I'd estimate around ~20 new formats for the
> > drivers in existing mainline kernel, and probably adding 1 or 2 per
> > new driver).
> >
> > Personally I don't see the third option as being practical, and the
> > second doesn't really add anything that significant over the first as
> > you still need to know the register setup.
I fully agree about the third option. The second option could be tempted
to provide help configuring the CSI-2 receiver and userspace decoder in
a more generic way, but in practice I believe that the related code will
know the pixel format being transmitted anyway.
However, wouldn't the CSI-2 specification allow transmitting embedded
data in a different format than pixel data ? That's probably not widely
done in practice, but if there's a chance it could happen, the second
option may be required.
> > You can't even easily take the second option and add read-only
> > controls telling userspace the registers for exposure and gain (and
> > anything else?). You'd also need the info on how many bits of each
> > register(s) are used, for exposure are all bits denoting whole lines,
> > or and do you compute the analogue gain code (this can often be
> > defined via 4 co-efficients, but that's not guaranteed). Perhaps it
> > can be described in a dozen new read only controls if you're lucky,
> > but then there's bound to be one sensor that can't.
> >
> > So where do you draw the line between too generic and totally
> > unscalable? Over to you.
>
> So a proper description of this format would be that it is custom sensor
> specific data stored in the first line of the image sent by the sensor.
First line or first few lines. There's also more non-image data that can
be sent by sensors, including PDAF data or statistics.
> Since this is sensor specific, how does libcamera know which sensor it is?
> It uses the sensor entity information for that?
Correct, libcamera identifies sensors, and has sensor-specific code.
Doing so through the entity name isn't ideal though, as the format of
the name has never been standardized :-/ It would have been nice to have
an entity model field, or a standardized format for the entity name.
> >>> +
> >>> +The size of the embedded buffer is defined as a single line with a pixel width
> >>> +width specified in bytes. This is obtained by a call to the
> >>
> >> "pixel width width"? Typo?
> >>
> >>> +:c:type:`VIDIOC_SUBDEV_G_FMT` ioctl on the sensor subdevice where the ``pad``
> >>> +field in :c:type:`v4l2_subdev_format` is set to 1. Note that this size is fixed
> >>
> >> Should this description be explicit about the pad number?
Work is ongoing to try and use the proposed multiplexed streams API to
avoid exposing two pads.
> >>> +and cannot be modified with a call to :c:type:`VIDIOC_SUBDEV_S_FMT`.
> >>> +
> >>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>> index b2ef8e60ea7d..faf5a0f5eb6b 100644
> >>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>> @@ -1346,6 +1346,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >>> case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata"; break;
> >>> case V4L2_META_FMT_D4XX: descr = "Intel D4xx UVC Metadata"; break;
> >>> case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
> >>> + case V4L2_META_FMT_SENSOR_DATA: descr = "Sensor Ancillary Metadata"; break;
> >>>
> >>> default:
> >>> /* Compressed formats */
> >>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >>> index 9817b7e2c968..a96146223843 100644
> >>> --- a/include/uapi/linux/videodev2.h
> >>> +++ b/include/uapi/linux/videodev2.h
> >>> @@ -766,6 +766,7 @@ struct v4l2_pix_format {
> >>> #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> >>> #define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> >>> #define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
> >>> +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S') /* Sensor Ancillary metadata */
> >>>
> >>> /* priv field value to indicates that subsequent fields are valid. */
> >>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type
2020-05-04 16:08 ` Laurent Pinchart
@ 2020-05-05 11:20 ` Dave Stevenson
0 siblings, 0 replies; 111+ messages in thread
From: Dave Stevenson @ 2020-05-05 11:20 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Hans Verkuil, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
Hi Laurent and Hans
On Mon, 4 May 2020 at 17:08, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Hans,
>
> On Mon, May 04, 2020 at 05:32:42PM +0200, Hans Verkuil wrote:
> > On 04/05/2020 16:39, Dave Stevenson wrote:
> > > On Mon, 4 May 2020 at 14:48, Hans Verkuil <hverkuil@xs4all.nl> wrote:
> > >> On 04/05/2020 11:25, Laurent Pinchart wrote:
> > >>> From: Naushir Patuck <naush@raspberrypi.com>
> > >>>
> > >>> Add V4L2_META_FMT_SENSOR_DATA format 4CC.
> > >>>
> > >>> This new format will be used by the BCM2835 Unicam device to return
> > >>> out camera sensor embedded data.
> > >>
> > >> out -> our (I think)
> > >
> > > More like drop the word totally.
> > > "used by BCM2835 Unicam device to return camera sensor embedded data"
> > >
> > > The reference to Unicam probably ought to be dropped as other SoCs may
> > > also be able to return embedded data.
> > >
> > >>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > >>> ---
> > >>> Documentation/media/uapi/v4l/meta-formats.rst | 1 +
> > >>> .../uapi/v4l/pixfmt-meta-sensor-data.rst | 32 +++++++++++++++++++
> > >>> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> > >>> include/uapi/linux/videodev2.h | 1 +
> > >>> 4 files changed, 35 insertions(+)
> > >>> create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> > >>>
> > >>> diff --git a/Documentation/media/uapi/v4l/meta-formats.rst b/Documentation/media/uapi/v4l/meta-formats.rst
> > >>> index 74c8659ee9d6..5474086ef6f0 100644
> > >>> --- a/Documentation/media/uapi/v4l/meta-formats.rst
> > >>> +++ b/Documentation/media/uapi/v4l/meta-formats.rst
> > >>> @@ -21,6 +21,7 @@ These formats are used for the :ref:`metadata` interface only.
> > >>>
> > >>> pixfmt-meta-d4xx
> > >>> pixfmt-meta-intel-ipu3
> > >>> + pixfmt-meta-sensor-data
> > >>> pixfmt-meta-uvc
> > >>> pixfmt-meta-vsp1-hgo
> > >>> pixfmt-meta-vsp1-hgt
> > >>> diff --git a/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> > >>> new file mode 100644
> > >>> index 000000000000..4a67e204d08a
> > >>> --- /dev/null
> > >>> +++ b/Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> > >>> @@ -0,0 +1,32 @@
> > >>> +.. Permission is granted to copy, distribute and/or modify this
> > >>> +.. document under the terms of the GNU Free Documentation License,
> > >>> +.. Version 1.1 or any later version published by the Free Software
> > >>> +.. Foundation, with no Invariant Sections, no Front-Cover Texts
> > >>> +.. and no Back-Cover Texts. A copy of the license is included at
> > >>> +.. Documentation/media/uapi/fdl-appendix.rst.
> > >>> +..
> > >>> +.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
> > >>> +
> > >>> +.. _v4l2-meta-fmt-sensor-data:
> > >>> +
> > >>> +***********************************
> > >>> +V4L2_META_FMT_SENSOR_DATA ('SENS')
> > >>> +***********************************
> > >>> +
> > >>> +Sensor Ancillary Metadata
> > >>> +
> > >>> +Description
> > >>> +===========
> > >>> +
> > >>> +This format describes ancillary data generated by a camera sensor and
> > >>> +transmitted over a stream on the camera bus. Sensor vendors generally have their
> > >>
> > >> 'generally' can be dropped.
> > >>
> > >>> +own custom format for this ancillary data. Some vendors follow a generic
> > >>> +CSI-2/SMIA embedded data format as described in the `CSI-2 specification.
> > >>> +<https://mipi.org/specifications/csi-2>`_
> > >>
> > >> So what does this format represent? Sensor specific custom data where you need
> > >> sensor-specific NDA knowledge in userspace to parse it? Or SMIA formatted sensor
> > >> data?
> > >>
> > >> A 4cc should map to a specific data format. I understand that this is not always
> > >> known, but at least the SMIA data format would need its own 4cc. I assume that the
> > >> sensor driver will at least know that it produces SMIA data, right?
> > >>
> > >> "SENSOR_DATA" is also much too generic IMHO.
> > >
> > > It's a tricky one to call as to either creating a vast number of
> > > formats vs being too generic.
> > >
> > > With V4L2 setting exposure as a number of lines, and analogue gain as
> > > the precomputed the gain code, decoding the register set from embedded
> > > data almost always requires sensor specific information.
> > >
> > > SMIA defines a strict register set. Yes you could decode that using
> > > the defined rules. However I'm not aware of any current sensors that
> > > actually implement the SMIA spec - that pretty much died with Nokia.
> >
> > OK.
>
> Two points worth noting here:
>
> - SMIA and SMIA++ indeed died with Nokia, but quite a few sensors are
> still at least partly compatible with the specs, as sensor vendors
> have a tendency to start development of new sensors based on older
> hardware implementations. I don't know to what extent this applies to
> metadata, or to recent sensors.
>
> - MIPI has released the Camera Command Set (CCS) specification
> (https://www.mipi.org/specifications/camera-command-set), which is
> similar to SMIA++ for the command set part. Register offsets are
> different, but the sensor model and exposed data is similar. The
> specification standardizes the format of embedded data, and in
> practice the data we're interested in is expressed as a set of
> tag/value pairs. Tags are more or less register addresses, and I think
> vendor-specific tags are allowed (as vendor-specific registers are
> also allowed). The specification isn't widely implemented yet, but
> some sensors that are not advertized as CCS-compliant actually follow
> the specification to some extent (again with more or less variation
> depending on the sensor model).
>
> The CCS specification is public, you can check it out.
>
> > > Looking slightly more generically, there is the encoding of the
> > > embedded data based on the Bayer format
> > > Raw 10 puts a dummy byte in every 5th byte to "match" the Bayer
> > > encoding of the least sig bits for 4 pixels being packed in the 5th
> > > byte.
> > > Raw 12 puts a dummy byte in every 3rd byte to "match" the Bayer raw 12 format.
> > > Raw 14 I have no idea over with 3 pixels in 7 bytes packing.
> > > (Raw 8 has no additional bytes, and I wouldn't expect raw 16 to have
> > > any either).
>
> There are two options in RAW16, efficient packing with 2 bytes per
> sample, or legacy packing with one byte per sample.
Thanks - I've never had the need to look as I haven't encountered a
16bit sensor in the flesh, but useful to know.
> > > This could be defined as a SMIA-style packing, and is very common.
> > > They also tend to use the same 0x55 0x5a 0xa5 0xaa values for
> > > signalling for which registers are used number is being presented.
> > >
> > > So you could reasonably define a SENSOR_DATA_RAW8, SENSOR_DATA_RAW10,
> > > SENSOR_DATA_RAW12, SENSOR_DATA_RAW14, and SENSOR_DATA_RAW16 formats,
> > > but userland still needs to know in which registers the actual useful
> > > values are.
> > > You've also now mandated support of V4L2_EVENT_SOURCE_CHANGE events,
> > > as changing the Bayer format on the video node will change the
> > > metadata format on the other.
> > >
> > > However unless you also know the exact sensor, you still don't know
> > > which register(s) relates to exposure or analogue gain. To do that
> > > would imply a unique 4CC in both mediabus and V4L2 domains for every
> > > sensor that varies the register configuration (families of sensors may
> > > put them in the same place if you're lucky). And you'll need
> > > raw10/12/14 packing variants of each based on the formats supported by
> > > the sensor.
> >
> > Out of curiosity: is the description of this metadata still something
> > that is typically under NDA? Or is this opening up more?
>
> It varies between sensor vendors. For CCS-compliant sensors most of the
> information is in the spec. The vendor-specific registers are usually
> under NDA, but some vendors release datasheets, at least for some of
> their products (for instance Aptina, now On Semi, has traditionally been
> more open than OmniVision). I don't know if moving towards CCS
> compliance will have the undesired side effect of leaving
> vendor-specific registers not documented.
The formatting and padding characters are generally common to most
vendors that I've encountered. However it really is a dump of a large
proportion of the register set, and that is custom per vendor, if not
per sensor.
Seeing as it has been made public by someone, the IMX219 datasheet[1]
describes the embedded data lines in section 4-1-6. The formatting
described there is the bit that is generally common to most sensors.
With that generic information you can get the register number / value
pairs, but not the meaning.
Section 3-5 then describes which registers will be included in the
embedded data. That is the bit that will vary between sensors. SMIA
and CCS try to define these register meanings, but one is dead and the
other isn't widely adopted.
[1] https://github.com/rellimmot/Sony-IMX219-Raspberry-Pi-V2-CMOS/blob/master/RASPBERRY%20PI%20CAMERA%20V2%20DATASHEET%20IMX219PQH5_7.0.0_Datasheet_XXX.PDF
> > > So as I see it you have 3 levels:
> > > 1) Totally generic and let userspace have to worry about the problem.
> > > (1 new format)
> > > 2) Semi generic raw8/10/12/14/16 packing for the data, but still
> > > userspace needs to know sensor specifics to decode anything useful
> > > from it (5 new formats)
> > > 3) Unique formats for every sensor, and for each bit depth that that
> > > sensor can produce (I'd estimate around ~20 new formats for the
> > > drivers in existing mainline kernel, and probably adding 1 or 2 per
> > > new driver).
> > >
> > > Personally I don't see the third option as being practical, and the
> > > second doesn't really add anything that significant over the first as
> > > you still need to know the register setup.
>
> I fully agree about the third option. The second option could be tempted
> to provide help configuring the CSI-2 receiver and userspace decoder in
> a more generic way, but in practice I believe that the related code will
> know the pixel format being transmitted anyway.
>
> However, wouldn't the CSI-2 specification allow transmitting embedded
> data in a different format than pixel data ? That's probably not widely
> done in practice, but if there's a chance it could happen, the second
> option may be required.
There's nothing in the CSI-2 spec that enforces anything over formats
of data. Supposedly the data type code in the packet header tells you
the format of any video type, but I've never investigated as to
exactly which data type code embedded data typically uses (Unicam
differentiates between "defined image data type" and "any other
value").
> > > You can't even easily take the second option and add read-only
> > > controls telling userspace the registers for exposure and gain (and
> > > anything else?). You'd also need the info on how many bits of each
> > > register(s) are used, for exposure are all bits denoting whole lines,
> > > or and do you compute the analogue gain code (this can often be
> > > defined via 4 co-efficients, but that's not guaranteed). Perhaps it
> > > can be described in a dozen new read only controls if you're lucky,
> > > but then there's bound to be one sensor that can't.
> > >
> > > So where do you draw the line between too generic and totally
> > > unscalable? Over to you.
> >
> > So a proper description of this format would be that it is custom sensor
> > specific data stored in the first line of the image sent by the sensor.
>
> First line or first few lines. There's also more non-image data that can
> be sent by sensors, including PDAF data or statistics.
It's not lines of the image as it is on a different CSI2 data type,
but it is generally referred to as being "lines" (just to be
confusing).
And yes you can get all sorts of other stuff muxed in with different
data type values. For example the Toshiba TC358743 can send audio
and/or infoframes down on an alternate data types, but there I would
agree it would be necessary to define new pixel formats as it is
definitely not SENSOR_DATA (I am not entertaining the idea of
implementing it - I2S is a better solution for the audio, and I2C is
sufficient for most InfoFrames).
Dave
> > Since this is sensor specific, how does libcamera know which sensor it is?
> > It uses the sensor entity information for that?
>
> Correct, libcamera identifies sensors, and has sensor-specific code.
> Doing so through the entity name isn't ideal though, as the format of
> the name has never been standardized :-/ It would have been nice to have
> an entity model field, or a standardized format for the entity name.
>
> > >>> +
> > >>> +The size of the embedded buffer is defined as a single line with a pixel width
> > >>> +width specified in bytes. This is obtained by a call to the
> > >>
> > >> "pixel width width"? Typo?
> > >>
> > >>> +:c:type:`VIDIOC_SUBDEV_G_FMT` ioctl on the sensor subdevice where the ``pad``
> > >>> +field in :c:type:`v4l2_subdev_format` is set to 1. Note that this size is fixed
> > >>
> > >> Should this description be explicit about the pad number?
>
> Work is ongoing to try and use the proposed multiplexed streams API to
> avoid exposing two pads.
>
> > >>> +and cannot be modified with a call to :c:type:`VIDIOC_SUBDEV_S_FMT`.
> > >>> +
> > >>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > >>> index b2ef8e60ea7d..faf5a0f5eb6b 100644
> > >>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > >>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > >>> @@ -1346,6 +1346,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> > >>> case V4L2_META_FMT_UVC: descr = "UVC Payload Header Metadata"; break;
> > >>> case V4L2_META_FMT_D4XX: descr = "Intel D4xx UVC Metadata"; break;
> > >>> case V4L2_META_FMT_VIVID: descr = "Vivid Metadata"; break;
> > >>> + case V4L2_META_FMT_SENSOR_DATA: descr = "Sensor Ancillary Metadata"; break;
> > >>>
> > >>> default:
> > >>> /* Compressed formats */
> > >>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > >>> index 9817b7e2c968..a96146223843 100644
> > >>> --- a/include/uapi/linux/videodev2.h
> > >>> +++ b/include/uapi/linux/videodev2.h
> > >>> @@ -766,6 +766,7 @@ struct v4l2_pix_format {
> > >>> #define V4L2_META_FMT_UVC v4l2_fourcc('U', 'V', 'C', 'H') /* UVC Payload Header metadata */
> > >>> #define V4L2_META_FMT_D4XX v4l2_fourcc('D', '4', 'X', 'X') /* D4XX Payload Header metadata */
> > >>> #define V4L2_META_FMT_VIVID v4l2_fourcc('V', 'I', 'V', 'D') /* Vivid Metadata */
> > >>> +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S') /* Sensor Ancillary metadata */
> > >>>
> > >>> /* priv field value to indicates that subsequent fields are valid. */
> > >>> #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
>
> --
> Regards,
>
> Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 02/34] media: uapi: Add MEDIA_BUS_FMT_SENSOR_DATA media bus format
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 03/34] dt-bindings: media: Document BCM283x CSI2/CCP2 receiver Laurent Pinchart
` (32 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Naushir Patuck <naush@raspberrypi.com>
This patch adds MEDIA_BUS_FMT_SENSOR_DATA used by the bcm2835-unicam
driver to support CSI-2 embedded data streams from camera sensors.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
---
.../media/uapi/v4l/subdev-formats.rst | 33 +++++++++++++++++++
include/uapi/linux/media-bus-format.h | 3 ++
2 files changed, 36 insertions(+)
diff --git a/Documentation/media/uapi/v4l/subdev-formats.rst b/Documentation/media/uapi/v4l/subdev-formats.rst
index 17bfb2beaa6a..e79caa61b9ce 100644
--- a/Documentation/media/uapi/v4l/subdev-formats.rst
+++ b/Documentation/media/uapi/v4l/subdev-formats.rst
@@ -7831,3 +7831,36 @@ formats.
- 0x5001
- Interleaved raw UYVY and JPEG image format with embedded meta-data
used by Samsung S3C73MX camera sensors.
+
+
+
+.. _v4l2-mbus-sensor-data:
+
+Sensor Ancillary Metadata Formats
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This section lists ancillary data generated by a camera sensor and
+transmitted over a stream on the camera bus.
+
+The following table lists the existing sensor ancillary metadata formats:
+
+
+.. _v4l2-mbus-pixelcode-sensor-metadata:
+
+.. tabularcolumns:: |p{8.0cm}|p{1.4cm}|p{7.7cm}|
+
+.. flat-table:: Sensor ancillary metadata formats
+ :header-rows: 1
+ :stub-columns: 0
+
+ * - Identifier
+ - Code
+ - Comments
+ * .. _MEDIA_BUS_FMT_SENSOR_DATA:
+
+ - MEDIA_BUS_FMT_SENSOR_DATA
+ - 0x7001
+ - Sensor vendor specific ancillary metadata. Some vendors follow a generic
+ CSI-2/SMIA embedded data format as described in the `CSI-2 specification.
+ <https://mipi.org/specifications/csi-2>`_
+
diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h
index 84fa53ffb13f..3c2848e91c1b 100644
--- a/include/uapi/linux/media-bus-format.h
+++ b/include/uapi/linux/media-bus-format.h
@@ -156,4 +156,7 @@
/* HSV - next is 0x6002 */
#define MEDIA_BUS_FMT_AHSV8888_1X32 0x6001
+/* Sensor ancillary metadata formats - next is 0x7002 */
+#define MEDIA_BUS_FMT_SENSOR_DATA 0x7001
+
#endif /* __LINUX_MEDIA_BUS_FORMAT_H */
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 03/34] dt-bindings: media: Document BCM283x CSI2/CCP2 receiver
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 01/34] media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 02/34] media: uapi: Add MEDIA_BUS_FMT_SENSOR_DATA media bus format Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface Laurent Pinchart
` (31 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.com>
Document the DT bindings for the CSI2/CCP2 receiver peripheral (known as
Unicam) on BCM283x SoCs.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
.../bindings/media/brcm,bcm2835-unicam.yaml | 155 ++++++++++++++++++
1 file changed, 155 insertions(+)
create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
diff --git a/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
new file mode 100644
index 000000000000..6ffc900e8ae8
--- /dev/null
+++ b/Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
@@ -0,0 +1,155 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+# Copyright (C) 2020 Raspberry Pi (Trading) Ltd.
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/brcm,bcm2835-unicam.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom BCM283x Camera Interface (Unicam)
+
+maintainers:
+ - Dave Stevenson <dave.stevenson@raspberrypi.com>
+ - Raspberry Pi kernel list <kernel-list@raspberrypi.com>
+
+description:
+ The Unicam block on BCM283x SoCs is the receiver for either CSI-2 or CCP2
+ data from image sensors or similar devices.
+
+ The main platform using this SoC is the Raspberry Pi family of boards. On the
+ Pi the VideoCore firmware can also control this hardware block, and driving
+ it from two different processors will cause issues. To avoid this, the
+ firmware checks the device tree configuration during boot. If it finds device
+ tree nodes whose name starts with "csi" then it will stop the firmware
+ accessing the block, and it can then safely be used via the device tree
+ binding.
+
+properties:
+ compatible:
+ const: brcm,bcm2835-unicam
+
+ reg:
+ items:
+ - description: Main registers block
+ - description: Clock registers block
+
+ interrupts:
+ maxItems: 1
+
+ clocks:
+ minItems: 1
+ maxItems: 2
+
+ clock-names:
+ minItems: 1
+ items:
+ - const: lp
+ - const: core
+
+ power-domains:
+ maxItems: 1
+
+ brcm,num-data-lanes:
+ description:
+ The number of data lanes supported by this Unicam instance. It may be
+ larger than the number of data lanes routed on the board, as described by
+ the data-lanes property of the endpoint.
+ allOf:
+ - $ref: "/schemas/types.yaml#/definitions/uint32"
+ - enum: [1, 2, 4]
+
+ port:
+ type: object
+ description:
+ Input port node, as described in video-interfaces.txt.
+
+ properties:
+ endpoint:
+ type: object
+
+ properties:
+ clock-lanes:
+ items:
+ - const: 0
+
+ data-lanes:
+ description:
+ Lane reordering is not supported, items shall be in order,
+ starting at 1.
+ allOf:
+ - $ref: "/schemas/types.yaml#/definitions/uint32-array"
+ - maxItems: 1
+ items:
+ minItems: 1
+ maxItems: 4
+ items:
+ - const: 1
+ - const: 2
+ - const: 3
+ - const: 4
+
+ lane-polarities:
+ description:
+ Lane inversion is not supported. If the property is specified, it
+ shall contain all 0's.
+ allOf:
+ - $ref: "/schemas/types.yaml#/definitions/uint32-array"
+ - maxItems: 1
+ items:
+ minItems: 2
+ maxItems: 5
+ items:
+ - const: 0
+ - const: 0
+ - const: 0
+ - const: 0
+ - const: 0
+
+ remote-endpoint: true
+
+ required:
+ - data-lanes
+ - remote-endpoint
+
+ additionalProperties: false
+
+ required:
+ - endpoint
+
+ additionalProperties: false
+
+required:
+ - compatible
+ - reg
+ - interrupts
+ - clocks
+ - clock-names
+ - power-domains
+ - brcm,num-data-lanes
+ - port
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/clock/bcm2835.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+ #include <dt-bindings/power/raspberrypi-power.h>
+
+ csi@7e801000 {
+ compatible = "brcm,bcm2835-unicam";
+ reg = <0x7e801000 0x800>,
+ <0x7e802004 0x4>;
+ interrupts = <2 7>;
+ clocks = <&clocks BCM2835_CLOCK_CAM1>;
+ clock-names = "lp";
+ power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;
+ brcm,num-data-lanes = <4>;
+
+ port {
+ csi1_ep: endpoint {
+ remote-endpoint = <&imx219_0>;
+ data-lanes = <1 2>;
+ };
+ };
+ };
+...
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (2 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 03/34] dt-bindings: media: Document BCM283x CSI2/CCP2 receiver Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 15:21 ` Hans Verkuil
` (4 more replies)
2020-05-04 9:25 ` [PATCH v2 05/34] ARM: dts: bcm2711: Add Unicam DT nodes Laurent Pinchart
` (30 subsequent siblings)
34 siblings, 5 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Naushir Patuck <naush@raspberrypi.com>
Add a driver for the Unicam camera receiver block on BCM283x processors.
Compared to the bcm2835-camera driver present in staging, this driver
handles the Unicam block only (CSI-2 receiver), and doesn't depend on
the VC4 firmware running on the VPU.
The commit is made up of a series of changes cherry-picked from the
rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
additional enhancements, forward-ported to the mainline kernel.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
Changes since v1:
- Re-fetch mbus code from subdev on a g_fmt call
- Group all ioctl disabling together
- Fix reference counting in unicam_open
- Add support for VIDIOC_[S|G]_SELECTION
---
MAINTAINERS | 7 +
drivers/media/platform/Kconfig | 1 +
drivers/media/platform/Makefile | 2 +
drivers/media/platform/bcm2835/Kconfig | 15 +
drivers/media/platform/bcm2835/Makefile | 3 +
.../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
.../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
7 files changed, 3106 insertions(+)
create mode 100644 drivers/media/platform/bcm2835/Kconfig
create mode 100644 drivers/media/platform/bcm2835/Makefile
create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
diff --git a/MAINTAINERS b/MAINTAINERS
index b816a453b10e..edf5b8d9c98a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3341,6 +3341,13 @@ N: bcm113*
N: bcm216*
N: kona
+BROADCOM BCM2835 CAMERA DRIVER
+M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
+L: linux-media@vger.kernel.org
+S: Maintained
+F: drivers/media/platform/bcm2835/
+F: Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
+
BROADCOM BCM47XX MIPS ARCHITECTURE
M: Hauke Mehrtens <hauke@hauke-m.de>
M: Rafał Miłecki <zajec5@gmail.com>
diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
index e01bbb9dd1c1..98721a4e0be1 100644
--- a/drivers/media/platform/Kconfig
+++ b/drivers/media/platform/Kconfig
@@ -146,6 +146,7 @@ source "drivers/media/platform/am437x/Kconfig"
source "drivers/media/platform/xilinx/Kconfig"
source "drivers/media/platform/rcar-vin/Kconfig"
source "drivers/media/platform/atmel/Kconfig"
+source "drivers/media/platform/bcm2835/Kconfig"
source "drivers/media/platform/sunxi/Kconfig"
config VIDEO_TI_CAL
diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
index d13db96e3015..a425e4d2e3f3 100644
--- a/drivers/media/platform/Makefile
+++ b/drivers/media/platform/Makefile
@@ -98,4 +98,6 @@ obj-y += meson/
obj-y += cros-ec-cec/
+obj-y += bcm2835/
+
obj-y += sunxi/
diff --git a/drivers/media/platform/bcm2835/Kconfig b/drivers/media/platform/bcm2835/Kconfig
new file mode 100644
index 000000000000..ec46e3ef053c
--- /dev/null
+++ b/drivers/media/platform/bcm2835/Kconfig
@@ -0,0 +1,15 @@
+# Broadcom VideoCore4 V4L2 camera support
+
+config VIDEO_BCM2835_UNICAM
+ tristate "Broadcom BCM2835 Unicam video capture driver"
+ depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
+ depends on ARCH_BCM2835 || COMPILE_TEST
+ select VIDEOBUF2_DMA_CONTIG
+ select V4L2_FWNODE
+ help
+ Say Y here to enable support for the BCM2835 CSI-2 receiver. This is a
+ V4L2 driver that controls the CSI-2 receiver directly, independently
+ from the VC4 firmware.
+
+ To compile this driver as a module, choose M here. The module will be
+ called bcm2835-unicam.
diff --git a/drivers/media/platform/bcm2835/Makefile b/drivers/media/platform/bcm2835/Makefile
new file mode 100644
index 000000000000..a98aba03598a
--- /dev/null
+++ b/drivers/media/platform/bcm2835/Makefile
@@ -0,0 +1,3 @@
+# Makefile for BCM2835 Unicam driver
+
+obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c b/drivers/media/platform/bcm2835/bcm2835-unicam.c
new file mode 100644
index 000000000000..2e9387cbc1e0
--- /dev/null
+++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
@@ -0,0 +1,2825 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * BCM2835 Unicam Capture Driver
+ *
+ * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
+ *
+ * Dave Stevenson <dave.stevenson@raspberrypi.com>
+ *
+ * Based on TI am437x driver by
+ * Benoit Parrot <bparrot@ti.com>
+ * Lad, Prabhakar <prabhakar.csengg@gmail.com>
+ *
+ * and TI CAL camera interface driver by
+ * Benoit Parrot <bparrot@ti.com>
+ *
+ *
+ * There are two camera drivers in the kernel for BCM283x - this one
+ * and bcm2835-camera (currently in staging).
+ *
+ * This driver directly controls the Unicam peripheral - there is no
+ * involvement with the VideoCore firmware. Unicam receives CSI-2 or
+ * CCP2 data and writes it into SDRAM.
+ * The only potential processing options are to repack Bayer data into an
+ * alternate format, and applying windowing.
+ * The repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P
+ * to V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12,
+ * but not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both
+ * formats where the relevant formats are defined, and will automatically
+ * configure the repacking as required.
+ * Support for windowing may be added later.
+ *
+ * It should be possible to connect this driver to any sensor with a
+ * suitable output interface and V4L2 subdevice driver.
+ *
+ * bcm2835-camera uses the VideoCore firmware to control the sensor,
+ * Unicam, ISP, and all tuner control loops. Fully processed frames are
+ * delivered to the driver by the firmware. It only has sensor drivers
+ * for Omnivision OV5647, and Sony IMX219 sensors.
+ *
+ * The two drivers are mutually exclusive for the same Unicam instance.
+ * The VideoCore firmware checks the device tree configuration during boot.
+ * If it finds device tree nodes called csi0 or csi1 it will block the
+ * firmware from accessing the peripheral, and bcm2835-camera will
+ * not be able to stream data.
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/init.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+#include <linux/pinctrl/consumer.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <linux/videodev2.h>
+
+#include <media/v4l2-common.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-dev.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-dv-timings.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/v4l2-fwnode.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "vc4-regs-unicam.h"
+
+#define UNICAM_MODULE_NAME "unicam"
+#define UNICAM_VERSION "0.1.0"
+
+static int debug;
+module_param(debug, int, 0644);
+MODULE_PARM_DESC(debug, "Debug level 0-3");
+
+#define unicam_dbg(level, dev, fmt, arg...) \
+ v4l2_dbg(level, debug, &(dev)->v4l2_dev, fmt, ##arg)
+#define unicam_info(dev, fmt, arg...) \
+ v4l2_info(&(dev)->v4l2_dev, fmt, ##arg)
+#define unicam_err(dev, fmt, arg...) \
+ v4l2_err(&(dev)->v4l2_dev, fmt, ##arg)
+
+/*
+ * To protect against a dodgy sensor driver never returning an error from
+ * enum_mbus_code, set a maximum index value to be used.
+ */
+#define MAX_ENUM_MBUS_CODE 128
+
+/*
+ * Stride is a 16 bit register, but also has to be a multiple of 32.
+ */
+#define BPL_ALIGNMENT 32
+#define MAX_BYTESPERLINE ((1 << 16) - BPL_ALIGNMENT)
+/*
+ * Max width is therefore determined by the max stride divided by
+ * the number of bits per pixel. Take 32bpp as a
+ * worst case.
+ * No imposed limit on the height, so adopt a square image for want
+ * of anything better.
+ */
+#define MAX_WIDTH (MAX_BYTESPERLINE / 4)
+#define MAX_HEIGHT MAX_WIDTH
+/* Define a nominal minimum image size */
+#define MIN_WIDTH 16
+#define MIN_HEIGHT 16
+/* Default size of the embedded buffer */
+#define UNICAM_EMBEDDED_SIZE 8192
+
+/*
+ * Size of the dummy buffer. Can be any size really, but the DMA
+ * allocation works in units of page sizes.
+ */
+#define DUMMY_BUF_SIZE (PAGE_SIZE)
+
+enum pad_types {
+ IMAGE_PAD,
+ METADATA_PAD,
+ MAX_NODES
+};
+
+/*
+ * struct unicam_fmt - Unicam media bus format information
+ * @pixelformat: V4L2 pixel format FCC identifier. 0 if n/a.
+ * @repacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
+ * out to 16bpp. 0 if n/a.
+ * @code: V4L2 media bus format code.
+ * @depth: Bits per pixel as delivered from the source.
+ * @csi_dt: CSI data type.
+ * @check_variants: Flag to denote that there are multiple mediabus formats
+ * still in the list that could match this V4L2 format.
+ */
+struct unicam_fmt {
+ u32 fourcc;
+ u32 repacked_fourcc;
+ u32 code;
+ u8 depth;
+ u8 csi_dt;
+ u8 check_variants;
+};
+
+static const struct unicam_fmt formats[] = {
+ /* YUV Formats */
+ {
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .code = MEDIA_BUS_FMT_YUYV8_2X8,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ .check_variants = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .code = MEDIA_BUS_FMT_UYVY8_2X8,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ .check_variants = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YVYU,
+ .code = MEDIA_BUS_FMT_YVYU8_2X8,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ .check_variants = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_VYUY,
+ .code = MEDIA_BUS_FMT_VYUY8_2X8,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ .check_variants = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .code = MEDIA_BUS_FMT_YUYV8_1X16,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ }, {
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .code = MEDIA_BUS_FMT_UYVY8_1X16,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YVYU,
+ .code = MEDIA_BUS_FMT_YVYU8_1X16,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ }, {
+ .fourcc = V4L2_PIX_FMT_VYUY,
+ .code = MEDIA_BUS_FMT_VYUY8_1X16,
+ .depth = 16,
+ .csi_dt = 0x1e,
+ }, {
+ /* RGB Formats */
+ .fourcc = V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
+ .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
+ .depth = 16,
+ .csi_dt = 0x22,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB565X, /* rrrrrggg gggbbbbb */
+ .code = MEDIA_BUS_FMT_RGB565_2X8_BE,
+ .depth = 16,
+ .csi_dt = 0x22
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB555, /* gggbbbbb arrrrrgg */
+ .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
+ .depth = 16,
+ .csi_dt = 0x21,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB555X, /* arrrrrgg gggbbbbb */
+ .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
+ .depth = 16,
+ .csi_dt = 0x21,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB24, /* rgb */
+ .code = MEDIA_BUS_FMT_RGB888_1X24,
+ .depth = 24,
+ .csi_dt = 0x24,
+ }, {
+ .fourcc = V4L2_PIX_FMT_BGR24, /* bgr */
+ .code = MEDIA_BUS_FMT_BGR888_1X24,
+ .depth = 24,
+ .csi_dt = 0x24,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB32, /* argb */
+ .code = MEDIA_BUS_FMT_ARGB8888_1X32,
+ .depth = 32,
+ .csi_dt = 0x0,
+ }, {
+ /* Bayer Formats */
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ .code = MEDIA_BUS_FMT_SBGGR8_1X8,
+ .depth = 8,
+ .csi_dt = 0x2a,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ .code = MEDIA_BUS_FMT_SGBRG8_1X8,
+ .depth = 8,
+ .csi_dt = 0x2a,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ .code = MEDIA_BUS_FMT_SGRBG8_1X8,
+ .depth = 8,
+ .csi_dt = 0x2a,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ .code = MEDIA_BUS_FMT_SRGGB8_1X8,
+ .depth = 8,
+ .csi_dt = 0x2a,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR10P,
+ .repacked_fourcc = V4L2_PIX_FMT_SBGGR10,
+ .code = MEDIA_BUS_FMT_SBGGR10_1X10,
+ .depth = 10,
+ .csi_dt = 0x2b,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG10P,
+ .repacked_fourcc = V4L2_PIX_FMT_SGBRG10,
+ .code = MEDIA_BUS_FMT_SGBRG10_1X10,
+ .depth = 10,
+ .csi_dt = 0x2b,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG10P,
+ .repacked_fourcc = V4L2_PIX_FMT_SGRBG10,
+ .code = MEDIA_BUS_FMT_SGRBG10_1X10,
+ .depth = 10,
+ .csi_dt = 0x2b,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB10P,
+ .repacked_fourcc = V4L2_PIX_FMT_SRGGB10,
+ .code = MEDIA_BUS_FMT_SRGGB10_1X10,
+ .depth = 10,
+ .csi_dt = 0x2b,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR12P,
+ .repacked_fourcc = V4L2_PIX_FMT_SBGGR12,
+ .code = MEDIA_BUS_FMT_SBGGR12_1X12,
+ .depth = 12,
+ .csi_dt = 0x2c,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG12P,
+ .repacked_fourcc = V4L2_PIX_FMT_SGBRG12,
+ .code = MEDIA_BUS_FMT_SGBRG12_1X12,
+ .depth = 12,
+ .csi_dt = 0x2c,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG12P,
+ .repacked_fourcc = V4L2_PIX_FMT_SGRBG12,
+ .code = MEDIA_BUS_FMT_SGRBG12_1X12,
+ .depth = 12,
+ .csi_dt = 0x2c,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB12P,
+ .repacked_fourcc = V4L2_PIX_FMT_SRGGB12,
+ .code = MEDIA_BUS_FMT_SRGGB12_1X12,
+ .depth = 12,
+ .csi_dt = 0x2c,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR14P,
+ .code = MEDIA_BUS_FMT_SBGGR14_1X14,
+ .depth = 14,
+ .csi_dt = 0x2d,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG14P,
+ .code = MEDIA_BUS_FMT_SGBRG14_1X14,
+ .depth = 14,
+ .csi_dt = 0x2d,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG14P,
+ .code = MEDIA_BUS_FMT_SGRBG14_1X14,
+ .depth = 14,
+ .csi_dt = 0x2d,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SRGGB14P,
+ .code = MEDIA_BUS_FMT_SRGGB14_1X14,
+ .depth = 14,
+ .csi_dt = 0x2d,
+ }, {
+ /*
+ * 16 bit Bayer formats could be supported, but there is no CSI2
+ * data_type defined for raw 16, and no sensors that produce it at
+ * present.
+ */
+
+ /* Greyscale formats */
+ .fourcc = V4L2_PIX_FMT_GREY,
+ .code = MEDIA_BUS_FMT_Y8_1X8,
+ .depth = 8,
+ .csi_dt = 0x2a,
+ }, {
+ .fourcc = V4L2_PIX_FMT_Y10P,
+ .repacked_fourcc = V4L2_PIX_FMT_Y10,
+ .code = MEDIA_BUS_FMT_Y10_1X10,
+ .depth = 10,
+ .csi_dt = 0x2b,
+ }, {
+ /* NB There is no packed V4L2 fourcc for this format. */
+ .repacked_fourcc = V4L2_PIX_FMT_Y12,
+ .code = MEDIA_BUS_FMT_Y12_1X12,
+ .depth = 12,
+ .csi_dt = 0x2c,
+ },
+ /* Embedded data format */
+ {
+ .fourcc = V4L2_META_FMT_SENSOR_DATA,
+ .code = MEDIA_BUS_FMT_SENSOR_DATA,
+ .depth = 8,
+ }
+};
+
+struct unicam_buffer {
+ struct vb2_v4l2_buffer vb;
+ struct list_head list;
+};
+
+static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
+{
+ return container_of(vb, struct unicam_buffer, vb.vb2_buf);
+}
+
+struct unicam_node {
+ bool registered;
+ int open;
+ bool streaming;
+ unsigned int pad_id;
+ /* Pointer pointing to current v4l2_buffer */
+ struct unicam_buffer *cur_frm;
+ /* Pointer pointing to next v4l2_buffer */
+ struct unicam_buffer *next_frm;
+ /* video capture */
+ const struct unicam_fmt *fmt;
+ /* Used to store current pixel format */
+ struct v4l2_format v_fmt;
+ /* Used to store current mbus frame format */
+ struct v4l2_mbus_framefmt m_fmt;
+ /* Buffer queue used in video-buf */
+ struct vb2_queue buffer_queue;
+ /* Queue of filled frames */
+ struct list_head dma_queue;
+ /* IRQ lock for DMA queue */
+ spinlock_t dma_queue_lock;
+ /* lock used to access this structure */
+ struct mutex lock;
+ /* Identifies video device for this channel */
+ struct video_device video_dev;
+ /* Pointer to the parent handle */
+ struct unicam_device *dev;
+ struct media_pad pad;
+ unsigned int embedded_lines;
+ /*
+ * Dummy buffer intended to be used by unicam
+ * if we have no other queued buffers to swap to.
+ */
+ void *dummy_buf_cpu_addr;
+ dma_addr_t dummy_buf_dma_addr;
+};
+
+struct unicam_device {
+ struct kref kref;
+
+ /* V4l2 specific parameters */
+ struct v4l2_async_subdev asd;
+
+ /* peripheral base address */
+ void __iomem *base;
+ /* clock gating base address */
+ void __iomem *clk_gate_base;
+ /* clock handle */
+ struct clk *clock;
+ /* V4l2 device */
+ struct v4l2_device v4l2_dev;
+ struct media_device mdev;
+
+ /* parent device */
+ struct platform_device *pdev;
+ /* subdevice async Notifier */
+ struct v4l2_async_notifier notifier;
+ unsigned int sequence;
+
+ /* ptr to sub device */
+ struct v4l2_subdev *sensor;
+ /* Pad config for the sensor */
+ struct v4l2_subdev_pad_config *sensor_config;
+
+ enum v4l2_mbus_type bus_type;
+ /*
+ * Stores bus.mipi_csi2.flags for CSI2 sensors, or
+ * bus.mipi_csi1.strobe for CCP2.
+ */
+ unsigned int bus_flags;
+ unsigned int max_data_lanes;
+ unsigned int active_data_lanes;
+ bool sensor_embedded_data;
+
+ struct unicam_node node[MAX_NODES];
+ struct v4l2_ctrl_handler ctrl_handler;
+};
+
+static inline struct unicam_device *
+to_unicam_device(struct v4l2_device *v4l2_dev)
+{
+ return container_of(v4l2_dev, struct unicam_device, v4l2_dev);
+}
+
+/* Hardware access */
+static inline void clk_write(struct unicam_device *dev, u32 val)
+{
+ writel(val | 0x5a000000, dev->clk_gate_base);
+}
+
+static inline u32 reg_read(struct unicam_device *dev, u32 offset)
+{
+ return readl(dev->base + offset);
+}
+
+static inline void reg_write(struct unicam_device *dev, u32 offset, u32 val)
+{
+ writel(val, dev->base + offset);
+}
+
+static inline int get_field(u32 value, u32 mask)
+{
+ return (value & mask) >> __ffs(mask);
+}
+
+static inline void set_field(u32 *valp, u32 field, u32 mask)
+{
+ u32 val = *valp;
+
+ val &= ~mask;
+ val |= (field << __ffs(mask)) & mask;
+ *valp = val;
+}
+
+static inline u32 reg_read_field(struct unicam_device *dev, u32 offset,
+ u32 mask)
+{
+ return get_field(reg_read(dev, offset), mask);
+}
+
+static inline void reg_write_field(struct unicam_device *dev, u32 offset,
+ u32 field, u32 mask)
+{
+ u32 val = reg_read(dev, offset);
+
+ set_field(&val, field, mask);
+ reg_write(dev, offset, val);
+}
+
+/* Power management functions */
+static inline int unicam_runtime_get(struct unicam_device *dev)
+{
+ return pm_runtime_get_sync(&dev->pdev->dev);
+}
+
+static inline void unicam_runtime_put(struct unicam_device *dev)
+{
+ pm_runtime_put_sync(&dev->pdev->dev);
+}
+
+/* Format setup functions */
+static const struct unicam_fmt *find_format_by_code(u32 code)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(formats); i++) {
+ if (formats[i].code == code)
+ return &formats[i];
+ }
+
+ return NULL;
+}
+
+static int check_mbus_format(struct unicam_device *dev,
+ const struct unicam_fmt *format)
+{
+ unsigned int i;
+ int ret = 0;
+
+ for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
+ struct v4l2_subdev_mbus_code_enum mbus_code = {
+ .index = i,
+ .pad = IMAGE_PAD,
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ };
+
+ ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
+ NULL, &mbus_code);
+
+ if (!ret && mbus_code.code == format->code)
+ return 1;
+ }
+
+ return 0;
+}
+
+static const struct unicam_fmt *find_format_by_pix(struct unicam_device *dev,
+ u32 pixelformat)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(formats); i++) {
+ if (formats[i].fourcc == pixelformat ||
+ formats[i].repacked_fourcc == pixelformat) {
+ if (formats[i].check_variants &&
+ !check_mbus_format(dev, &formats[i]))
+ continue;
+ return &formats[i];
+ }
+ }
+
+ return NULL;
+}
+
+static inline unsigned int bytes_per_line(u32 width,
+ const struct unicam_fmt *fmt,
+ u32 v4l2_fourcc)
+{
+ if (v4l2_fourcc == fmt->repacked_fourcc)
+ /* Repacking always goes to 16bpp */
+ return ALIGN(width << 1, BPL_ALIGNMENT);
+ else
+ return ALIGN((width * fmt->depth) >> 3, BPL_ALIGNMENT);
+}
+
+static int __subdev_get_format(struct unicam_device *dev,
+ struct v4l2_mbus_framefmt *fmt, int pad_id)
+{
+ struct v4l2_subdev_format sd_fmt = {
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .pad = pad_id
+ };
+ int ret;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, get_fmt, dev->sensor_config,
+ &sd_fmt);
+ if (ret < 0)
+ return ret;
+
+ *fmt = sd_fmt.format;
+
+ unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__,
+ fmt->width, fmt->height, fmt->code);
+
+ return 0;
+}
+
+static int __subdev_set_format(struct unicam_device *dev,
+ struct v4l2_mbus_framefmt *fmt, int pad_id)
+{
+ struct v4l2_subdev_format sd_fmt = {
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .pad = pad_id
+ };
+ int ret;
+
+ sd_fmt.format = *fmt;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev->sensor_config,
+ &sd_fmt);
+ if (ret < 0)
+ return ret;
+
+ *fmt = sd_fmt.format;
+
+ if (pad_id == IMAGE_PAD)
+ unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__, fmt->width,
+ fmt->height, fmt->code);
+ else
+ unicam_dbg(1, dev, "%s Embedded data code:%04x\n", __func__,
+ sd_fmt.format.code);
+
+ return 0;
+}
+
+static int unicam_calc_format_size_bpl(struct unicam_device *dev,
+ const struct unicam_fmt *fmt,
+ struct v4l2_format *f)
+{
+ unsigned int min_bytesperline;
+
+ v4l_bound_align_image(&f->fmt.pix.width, MIN_WIDTH, MAX_WIDTH, 2,
+ &f->fmt.pix.height, MIN_HEIGHT, MAX_HEIGHT, 0,
+ 0);
+
+ min_bytesperline = bytes_per_line(f->fmt.pix.width, fmt,
+ f->fmt.pix.pixelformat);
+
+ if (f->fmt.pix.bytesperline > min_bytesperline &&
+ f->fmt.pix.bytesperline <= MAX_BYTESPERLINE)
+ f->fmt.pix.bytesperline = ALIGN(f->fmt.pix.bytesperline,
+ BPL_ALIGNMENT);
+ else
+ f->fmt.pix.bytesperline = min_bytesperline;
+
+ f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+
+ unicam_dbg(3, dev, "%s: fourcc: %08X size: %dx%d bpl:%d img_size:%d\n",
+ __func__,
+ f->fmt.pix.pixelformat,
+ f->fmt.pix.width, f->fmt.pix.height,
+ f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+ return 0;
+}
+
+static int unicam_reset_format(struct unicam_node *node)
+{
+ struct unicam_device *dev = node->dev;
+ struct v4l2_mbus_framefmt mbus_fmt;
+ int ret;
+
+ if (dev->sensor_embedded_data || node->pad_id != METADATA_PAD) {
+ ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
+ if (ret) {
+ unicam_err(dev, "Failed to get_format - ret %d\n", ret);
+ return ret;
+ }
+
+ if (mbus_fmt.code != node->fmt->code) {
+ unicam_err(dev, "code mismatch - fmt->code %08x, mbus_fmt.code %08x\n",
+ node->fmt->code, mbus_fmt.code);
+ return ret;
+ }
+ }
+
+ if (node->pad_id == IMAGE_PAD) {
+ v4l2_fill_pix_format(&node->v_fmt.fmt.pix, &mbus_fmt);
+ node->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ unicam_calc_format_size_bpl(dev, node->fmt, &node->v_fmt);
+ } else {
+ node->v_fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
+ node->v_fmt.fmt.meta.dataformat = V4L2_META_FMT_SENSOR_DATA;
+ if (dev->sensor_embedded_data) {
+ node->v_fmt.fmt.meta.buffersize =
+ mbus_fmt.width * mbus_fmt.height;
+ node->embedded_lines = mbus_fmt.height;
+ } else {
+ node->v_fmt.fmt.meta.buffersize = UNICAM_EMBEDDED_SIZE;
+ node->embedded_lines = 1;
+ }
+ }
+
+ node->m_fmt = mbus_fmt;
+ return 0;
+}
+
+static void unicam_wr_dma_addr(struct unicam_device *dev, dma_addr_t dmaaddr,
+ unsigned int buffer_size, int pad_id)
+{
+ dma_addr_t endaddr = dmaaddr + buffer_size;
+
+ /*
+ * dmaaddr and endaddr should be a 32-bit address with the top two bits
+ * set to 0x3 to signify uncached access through the Videocore memory
+ * controller.
+ */
+ WARN_ON((dmaaddr >> 30) != 0x3 || (endaddr >> 30) != 0x3);
+
+ if (pad_id == IMAGE_PAD) {
+ reg_write(dev, UNICAM_IBSA0, dmaaddr);
+ reg_write(dev, UNICAM_IBEA0, endaddr);
+ } else {
+ reg_write(dev, UNICAM_DBSA0, dmaaddr);
+ reg_write(dev, UNICAM_DBEA0, endaddr);
+ }
+}
+
+static inline unsigned int unicam_get_lines_done(struct unicam_device *dev)
+{
+ dma_addr_t start_addr, cur_addr;
+ unsigned int stride = dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline;
+ struct unicam_buffer *frm = dev->node[IMAGE_PAD].cur_frm;
+
+ if (!frm)
+ return 0;
+
+ start_addr = vb2_dma_contig_plane_dma_addr(&frm->vb.vb2_buf, 0);
+ cur_addr = reg_read(dev, UNICAM_IBWP);
+ return (unsigned int)(cur_addr - start_addr) / stride;
+}
+
+static inline void unicam_schedule_next_buffer(struct unicam_node *node)
+{
+ struct unicam_device *dev = node->dev;
+ struct unicam_buffer *buf;
+ unsigned int size;
+ dma_addr_t addr;
+
+ buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
+ node->next_frm = buf;
+ list_del(&buf->list);
+
+ addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
+ size = (node->pad_id == IMAGE_PAD) ?
+ node->v_fmt.fmt.pix.sizeimage :
+ node->v_fmt.fmt.meta.buffersize;
+
+ unicam_wr_dma_addr(dev, addr, size, node->pad_id);
+}
+
+static inline void unicam_schedule_dummy_buffer(struct unicam_node *node)
+{
+ struct unicam_device *dev = node->dev;
+
+ unicam_dbg(3, dev, "Scheduling dummy buffer for node %d\n",
+ node->pad_id);
+
+ unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr, DUMMY_BUF_SIZE,
+ node->pad_id);
+ node->next_frm = NULL;
+}
+
+static inline void unicam_process_buffer_complete(struct unicam_node *node,
+ unsigned int sequence)
+{
+ node->cur_frm->vb.field = node->m_fmt.field;
+ node->cur_frm->vb.sequence = sequence;
+
+ vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
+}
+
+static bool unicam_all_nodes_streaming(struct unicam_device *dev)
+{
+ bool ret;
+
+ ret = dev->node[IMAGE_PAD].open && dev->node[IMAGE_PAD].streaming;
+ ret &= !dev->node[METADATA_PAD].open ||
+ dev->node[METADATA_PAD].streaming;
+ return ret;
+}
+
+static bool unicam_all_nodes_disabled(struct unicam_device *dev)
+{
+ return !dev->node[IMAGE_PAD].streaming &&
+ !dev->node[METADATA_PAD].streaming;
+}
+
+static void unicam_queue_event_sof(struct unicam_device *unicam)
+{
+ struct v4l2_event event = {
+ .type = V4L2_EVENT_FRAME_SYNC,
+ .u.frame_sync.frame_sequence = unicam->sequence,
+ };
+
+ v4l2_event_queue(&unicam->node[IMAGE_PAD].video_dev, &event);
+}
+
+/*
+ * unicam_isr : ISR handler for unicam capture
+ * @irq: irq number
+ * @dev_id: dev_id ptr
+ *
+ * It changes status of the captured buffer, takes next buffer from the queue
+ * and sets its address in unicam registers
+ */
+static irqreturn_t unicam_isr(int irq, void *dev)
+{
+ struct unicam_device *unicam = dev;
+ unsigned int lines_done = unicam_get_lines_done(dev);
+ unsigned int sequence = unicam->sequence;
+ unsigned int i;
+ u32 ista, sta;
+ u64 ts;
+
+ /*
+ * Don't service interrupts if not streaming.
+ * Avoids issues if the VPU should enable the
+ * peripheral without the kernel knowing (that
+ * shouldn't happen, but causes issues if it does).
+ */
+ if (unicam_all_nodes_disabled(unicam))
+ return IRQ_NONE;
+
+ sta = reg_read(unicam, UNICAM_STA);
+ /* Write value back to clear the interrupts */
+ reg_write(unicam, UNICAM_STA, sta);
+
+ ista = reg_read(unicam, UNICAM_ISTA);
+ /* Write value back to clear the interrupts */
+ reg_write(unicam, UNICAM_ISTA, ista);
+
+ unicam_dbg(3, unicam, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d",
+ ista, sta, sequence, lines_done);
+
+ if (!(sta & (UNICAM_IS | UNICAM_PI0)))
+ return IRQ_HANDLED;
+
+ /*
+ * We must run the frame end handler first. If we have a valid next_frm
+ * and we get a simultaneout FE + FS interrupt, running the FS handler
+ * first would null out the next_frm ptr and we would have lost the
+ * buffer forever.
+ */
+ if (ista & UNICAM_FEI || sta & UNICAM_PI0) {
+ /*
+ * Ensure we have swapped buffers already as we can't
+ * stop the peripheral. If no buffer is available, use a
+ * dummy buffer to dump out frames until we get a new buffer
+ * to use.
+ */
+ for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+ if (!unicam->node[i].streaming)
+ continue;
+
+ if (unicam->node[i].cur_frm)
+ unicam_process_buffer_complete(&unicam->node[i],
+ sequence);
+ unicam->node[i].cur_frm = unicam->node[i].next_frm;
+ }
+ unicam->sequence++;
+ }
+
+ if (ista & UNICAM_FSI) {
+ /*
+ * Timestamp is to be when the first data byte was captured,
+ * aka frame start.
+ */
+ ts = ktime_get_ns();
+ for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+ if (!unicam->node[i].streaming)
+ continue;
+
+ if (unicam->node[i].cur_frm)
+ unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
+ ts;
+ /*
+ * Set the next frame output to go to a dummy frame
+ * if we have not managed to obtain another frame
+ * from the queue.
+ */
+ unicam_schedule_dummy_buffer(&unicam->node[i]);
+ }
+
+ unicam_queue_event_sof(unicam);
+ }
+
+ /*
+ * Cannot swap buffer at frame end, there may be a race condition
+ * where the HW does not actually swap it if the new frame has
+ * already started.
+ */
+ if (ista & (UNICAM_FSI | UNICAM_LCI) && !(ista & UNICAM_FEI)) {
+ for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+ if (!unicam->node[i].streaming)
+ continue;
+
+ spin_lock(&unicam->node[i].dma_queue_lock);
+ if (!list_empty(&unicam->node[i].dma_queue) &&
+ !unicam->node[i].next_frm)
+ unicam_schedule_next_buffer(&unicam->node[i]);
+ spin_unlock(&unicam->node[i].dma_queue_lock);
+ }
+ }
+
+ if (reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
+ /* Switch out of trigger mode if selected */
+ reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
+ reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
+ }
+ return IRQ_HANDLED;
+}
+
+static int unicam_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ strlcpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
+ strlcpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
+
+ snprintf(cap->bus_info, sizeof(cap->bus_info),
+ "platform:%s", dev_name(&dev->pdev->dev));
+
+ cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
+
+ return 0;
+}
+
+static int unicam_enum_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ unsigned int index = 0;
+ unsigned int i;
+ int ret = 0;
+
+ if (node->pad_id != IMAGE_PAD)
+ return -EINVAL;
+
+ for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
+ struct v4l2_subdev_mbus_code_enum mbus_code = {
+ .index = i,
+ .pad = IMAGE_PAD,
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ };
+ const struct unicam_fmt *fmt;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
+ NULL, &mbus_code);
+ if (ret < 0) {
+ unicam_dbg(2, dev,
+ "subdev->enum_mbus_code idx %d returned %d - index invalid\n",
+ i, ret);
+ return -EINVAL;
+ }
+
+ fmt = find_format_by_code(mbus_code.code);
+ if (fmt) {
+ if (fmt->fourcc) {
+ if (index == f->index) {
+ f->pixelformat = fmt->fourcc;
+ break;
+ }
+ index++;
+ }
+ if (fmt->repacked_fourcc) {
+ if (index == f->index) {
+ f->pixelformat = fmt->repacked_fourcc;
+ break;
+ }
+ index++;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int unicam_g_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct v4l2_mbus_framefmt mbus_fmt = {0};
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ const struct unicam_fmt *fmt = NULL;
+ int ret;
+
+ if (node->pad_id != IMAGE_PAD)
+ return -EINVAL;
+
+ /*
+ * If a flip has occurred in the sensor, the fmt code might have
+ * changed. So we will need to re-fetch the format from the subdevice.
+ */
+ ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
+ if (ret)
+ return -EINVAL;
+
+ /* Find the V4L2 format from mbus code. We must match a known format. */
+ fmt = find_format_by_code(mbus_fmt.code);
+ if (!fmt)
+ return -EINVAL;
+
+ node->fmt = fmt;
+ node->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
+ *f = node->v_fmt;
+
+ return 0;
+}
+
+static
+const struct unicam_fmt *get_first_supported_format(struct unicam_device *dev)
+{
+ struct v4l2_subdev_mbus_code_enum mbus_code;
+ const struct unicam_fmt *fmt = NULL;
+ unsigned int i;
+ int ret;
+
+ for (i = 0; ret != -EINVAL && ret != -ENOIOCTLCMD; ++i) {
+ memset(&mbus_code, 0, sizeof(mbus_code));
+ mbus_code.index = i;
+ mbus_code.pad = IMAGE_PAD;
+ mbus_code.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code, NULL,
+ &mbus_code);
+ if (ret < 0) {
+ unicam_dbg(2, dev,
+ "subdev->enum_mbus_code idx %u returned %d - continue\n",
+ i, ret);
+ continue;
+ }
+
+ unicam_dbg(2, dev, "subdev %s: code: 0x%08x idx: %u\n",
+ dev->sensor->name, mbus_code.code, i);
+
+ fmt = find_format_by_code(mbus_code.code);
+ unicam_dbg(2, dev, "fmt 0x%08x returned as %p, V4L2 FOURCC 0x%08x, csi_dt 0x%02x\n",
+ mbus_code.code, fmt, fmt ? fmt->fourcc : 0,
+ fmt ? fmt->csi_dt : 0);
+ if (fmt)
+ return fmt;
+ }
+
+ return NULL;
+}
+
+static int unicam_try_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ struct v4l2_subdev_format sd_fmt = {
+ .which = V4L2_SUBDEV_FORMAT_TRY,
+ .pad = IMAGE_PAD
+ };
+ struct v4l2_mbus_framefmt *mbus_fmt = &sd_fmt.format;
+ const struct unicam_fmt *fmt;
+ int ret;
+
+ if (node->pad_id != IMAGE_PAD)
+ return -EINVAL;
+
+ fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
+ if (!fmt) {
+ /*
+ * Pixel format not supported by unicam. Choose the first
+ * supported format, and let the sensor choose something else.
+ */
+ unicam_dbg(3, dev, "Fourcc format (0x%08x) not found. Use first format.\n",
+ f->fmt.pix.pixelformat);
+
+ fmt = &formats[0];
+ f->fmt.pix.pixelformat = fmt->fourcc;
+ }
+
+ v4l2_fill_mbus_format(mbus_fmt, &f->fmt.pix, fmt->code);
+ /*
+ * No support for receiving interlaced video, so never
+ * request it from the sensor subdev.
+ */
+ mbus_fmt->field = V4L2_FIELD_NONE;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev->sensor_config,
+ &sd_fmt);
+ if (ret && ret != -ENOIOCTLCMD && ret != -ENODEV)
+ return ret;
+
+ if (mbus_fmt->field != V4L2_FIELD_NONE)
+ unicam_info(dev, "Sensor trying to send interlaced video - results may be unpredictable\n");
+
+ v4l2_fill_pix_format(&f->fmt.pix, &sd_fmt.format);
+ if (mbus_fmt->code != fmt->code) {
+ /* Sensor has returned an alternate format */
+ fmt = find_format_by_code(mbus_fmt->code);
+ if (!fmt) {
+ /*
+ * The alternate format is one unicam can't support.
+ * Find the first format that is supported by both, and
+ * then set that.
+ */
+ fmt = get_first_supported_format(dev);
+ mbus_fmt->code = fmt->code;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, set_fmt,
+ dev->sensor_config, &sd_fmt);
+ if (ret && ret != -ENOIOCTLCMD && ret != -ENODEV)
+ return ret;
+
+ if (mbus_fmt->field != V4L2_FIELD_NONE)
+ unicam_info(dev, "Sensor trying to send interlaced video - results may be unpredictable\n");
+
+ v4l2_fill_pix_format(&f->fmt.pix, &sd_fmt.format);
+
+ if (mbus_fmt->code != fmt->code) {
+ /*
+ * We've set a format that the sensor reports
+ * as being supported, but it refuses to set it.
+ * Not much else we can do.
+ * Assume that the sensor driver may accept the
+ * format when it is set (rather than tried).
+ */
+ unicam_err(dev, "Sensor won't accept default format, and Unicam can't support sensor default\n");
+ }
+ }
+
+ if (fmt->fourcc)
+ f->fmt.pix.pixelformat = fmt->fourcc;
+ else
+ f->fmt.pix.pixelformat = fmt->repacked_fourcc;
+ }
+
+ return unicam_calc_format_size_bpl(dev, fmt, f);
+}
+
+static int unicam_s_fmt_vid_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ struct vb2_queue *q = &node->buffer_queue;
+ struct v4l2_mbus_framefmt mbus_fmt = {0};
+ const struct unicam_fmt *fmt;
+ int ret;
+
+ if (vb2_is_busy(q))
+ return -EBUSY;
+
+ ret = unicam_try_fmt_vid_cap(file, priv, f);
+ if (ret < 0)
+ return ret;
+
+ fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
+ if (!fmt) {
+ /*
+ * Unknown pixel format - adopt a default.
+ * This shouldn't happen as try_fmt should have resolved any
+ * issues first.
+ */
+ fmt = get_first_supported_format(dev);
+ if (!fmt)
+ /*
+ * It shouldn't be possible to get here with no
+ * supported formats
+ */
+ return -EINVAL;
+ f->fmt.pix.pixelformat = fmt->fourcc;
+ return -EINVAL;
+ }
+
+ v4l2_fill_mbus_format(&mbus_fmt, &f->fmt.pix, fmt->code);
+
+ ret = __subdev_set_format(dev, &mbus_fmt, node->pad_id);
+ if (ret) {
+ unicam_dbg(3, dev, "%s __subdev_set_format failed %d\n",
+ __func__, ret);
+ return ret;
+ }
+
+ /* Just double check nothing has gone wrong */
+ if (mbus_fmt.code != fmt->code) {
+ unicam_dbg(3, dev,
+ "%s subdev changed format on us, this should not happen\n",
+ __func__);
+ return -EINVAL;
+ }
+
+ node->fmt = fmt;
+ node->v_fmt.fmt.pix.pixelformat = f->fmt.pix.pixelformat;
+ node->v_fmt.fmt.pix.bytesperline = f->fmt.pix.bytesperline;
+ unicam_reset_format(node);
+
+ unicam_dbg(3, dev,
+ "%s %dx%d, mbus_fmt 0x%08X, V4L2 pix 0x%08X.\n",
+ __func__, node->v_fmt.fmt.pix.width,
+ node->v_fmt.fmt.pix.height, mbus_fmt.code,
+ node->v_fmt.fmt.pix.pixelformat);
+
+ *f = node->v_fmt;
+
+ return 0;
+}
+
+static int unicam_enum_fmt_meta_cap(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ const struct unicam_fmt *fmt;
+ u32 code;
+ int ret = 0;
+
+ if (node->pad_id != METADATA_PAD || f->index != 0)
+ return -EINVAL;
+
+ if (dev->sensor_embedded_data) {
+ struct v4l2_subdev_mbus_code_enum mbus_code = {
+ .index = f->index,
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .pad = METADATA_PAD,
+ };
+
+ ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code, NULL,
+ &mbus_code);
+ if (ret < 0) {
+ unicam_dbg(2, dev,
+ "subdev->enum_mbus_code idx 0 returned %d - index invalid\n",
+ ret);
+ return -EINVAL;
+ }
+
+ code = mbus_code.code;
+ } else {
+ code = MEDIA_BUS_FMT_SENSOR_DATA;
+ }
+
+ fmt = find_format_by_code(code);
+ if (fmt)
+ f->pixelformat = fmt->fourcc;
+
+ return 0;
+}
+
+static int unicam_g_fmt_meta_cap(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct unicam_node *node = video_drvdata(file);
+
+ if (node->pad_id != METADATA_PAD)
+ return -EINVAL;
+
+ *f = node->v_fmt;
+
+ return 0;
+}
+
+static int unicam_queue_setup(struct vb2_queue *vq,
+ unsigned int *nbuffers,
+ unsigned int *nplanes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct unicam_node *node = vb2_get_drv_priv(vq);
+ struct unicam_device *dev = node->dev;
+ unsigned int size = node->pad_id == IMAGE_PAD ?
+ node->v_fmt.fmt.pix.sizeimage :
+ node->v_fmt.fmt.meta.buffersize;
+
+ if (vq->num_buffers + *nbuffers < 3)
+ *nbuffers = 3 - vq->num_buffers;
+
+ if (*nplanes) {
+ if (sizes[0] < size) {
+ unicam_err(dev, "sizes[0] %i < size %u\n", sizes[0],
+ size);
+ return -EINVAL;
+ }
+ size = sizes[0];
+ }
+
+ *nplanes = 1;
+ sizes[0] = size;
+
+ return 0;
+}
+
+static int unicam_buffer_prepare(struct vb2_buffer *vb)
+{
+ struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
+ struct unicam_device *dev = node->dev;
+ struct unicam_buffer *buf = to_unicam_buffer(vb);
+ unsigned long size;
+
+ if (WARN_ON(!node->fmt))
+ return -EINVAL;
+
+ size = node->pad_id == IMAGE_PAD ? node->v_fmt.fmt.pix.sizeimage :
+ node->v_fmt.fmt.meta.buffersize;
+ if (vb2_plane_size(vb, 0) < size) {
+ unicam_err(dev, "data will not fit into plane (%lu < %lu)\n",
+ vb2_plane_size(vb, 0), size);
+ return -EINVAL;
+ }
+
+ vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
+ return 0;
+}
+
+static void unicam_buffer_queue(struct vb2_buffer *vb)
+{
+ struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
+ struct unicam_buffer *buf = to_unicam_buffer(vb);
+ unsigned long flags;
+
+ spin_lock_irqsave(&node->dma_queue_lock, flags);
+ list_add_tail(&buf->list, &node->dma_queue);
+ spin_unlock_irqrestore(&node->dma_queue_lock, flags);
+}
+
+static void unicam_set_packing_config(struct unicam_device *dev)
+{
+ u32 pack, unpack;
+ u32 val;
+
+ if (dev->node[IMAGE_PAD].v_fmt.fmt.pix.pixelformat ==
+ dev->node[IMAGE_PAD].fmt->fourcc) {
+ unpack = UNICAM_PUM_NONE;
+ pack = UNICAM_PPM_NONE;
+ } else {
+ switch (dev->node[IMAGE_PAD].fmt->depth) {
+ case 8:
+ unpack = UNICAM_PUM_UNPACK8;
+ break;
+ case 10:
+ unpack = UNICAM_PUM_UNPACK10;
+ break;
+ case 12:
+ unpack = UNICAM_PUM_UNPACK12;
+ break;
+ case 14:
+ unpack = UNICAM_PUM_UNPACK14;
+ break;
+ case 16:
+ unpack = UNICAM_PUM_UNPACK16;
+ break;
+ default:
+ unpack = UNICAM_PUM_NONE;
+ break;
+ }
+
+ /* Repacking is always to 16bpp */
+ pack = UNICAM_PPM_PACK16;
+ }
+
+ val = 0;
+ set_field(&val, unpack, UNICAM_PUM_MASK);
+ set_field(&val, pack, UNICAM_PPM_MASK);
+ reg_write(dev, UNICAM_IPIPE, val);
+}
+
+static void unicam_cfg_image_id(struct unicam_device *dev)
+{
+ if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
+ /* CSI2 mode, hardcode VC 0 for now. */
+ reg_write(dev, UNICAM_IDI0,
+ (0 << 6) | dev->node[IMAGE_PAD].fmt->csi_dt);
+ } else {
+ /* CCP2 mode */
+ reg_write(dev, UNICAM_IDI0,
+ 0x80 | dev->node[IMAGE_PAD].fmt->csi_dt);
+ }
+}
+
+static void unicam_enable_ed(struct unicam_device *dev)
+{
+ u32 val = reg_read(dev, UNICAM_DCS);
+
+ set_field(&val, 2, UNICAM_EDL_MASK);
+ /* Do not wrap at the end of the embedded data buffer */
+ set_field(&val, 0, UNICAM_DBOB);
+
+ reg_write(dev, UNICAM_DCS, val);
+}
+
+static void unicam_start_rx(struct unicam_device *dev, dma_addr_t *addr)
+{
+ int line_int_freq = dev->node[IMAGE_PAD].v_fmt.fmt.pix.height >> 2;
+ unsigned int size, i;
+ u32 val;
+
+ if (line_int_freq < 128)
+ line_int_freq = 128;
+
+ /* Enable lane clocks */
+ val = 1;
+ for (i = 0; i < dev->active_data_lanes; i++)
+ val = val << 2 | 1;
+ clk_write(dev, val);
+
+ /* Basic init */
+ reg_write(dev, UNICAM_CTRL, UNICAM_MEM);
+
+ /* Enable analogue control, and leave in reset. */
+ val = UNICAM_AR;
+ set_field(&val, 7, UNICAM_CTATADJ_MASK);
+ set_field(&val, 7, UNICAM_PTATADJ_MASK);
+ reg_write(dev, UNICAM_ANA, val);
+ usleep_range(1000, 2000);
+
+ /* Come out of reset */
+ reg_write_field(dev, UNICAM_ANA, 0, UNICAM_AR);
+
+ /* Peripheral reset */
+ reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
+ reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
+
+ reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
+
+ /* Enable Rx control. */
+ val = reg_read(dev, UNICAM_CTRL);
+ if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
+ set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
+ set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
+ } else {
+ set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
+ set_field(&val, dev->bus_flags, UNICAM_DCM_MASK);
+ }
+ /* Packet framer timeout */
+ set_field(&val, 0xf, UNICAM_PFT_MASK);
+ set_field(&val, 128, UNICAM_OET_MASK);
+ reg_write(dev, UNICAM_CTRL, val);
+
+ reg_write(dev, UNICAM_IHWIN, 0);
+ reg_write(dev, UNICAM_IVWIN, 0);
+
+ /* AXI bus access QoS setup */
+ val = reg_read(dev, UNICAM_PRI);
+ set_field(&val, 0, UNICAM_BL_MASK);
+ set_field(&val, 0, UNICAM_BS_MASK);
+ set_field(&val, 0xe, UNICAM_PP_MASK);
+ set_field(&val, 8, UNICAM_NP_MASK);
+ set_field(&val, 2, UNICAM_PT_MASK);
+ set_field(&val, 1, UNICAM_PE);
+ reg_write(dev, UNICAM_PRI, val);
+
+ reg_write_field(dev, UNICAM_ANA, 0, UNICAM_DDL);
+
+ /* Always start in trigger frame capture mode (UNICAM_FCM set) */
+ val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
+ set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
+ reg_write(dev, UNICAM_ICTL, val);
+ reg_write(dev, UNICAM_STA, UNICAM_STA_MASK_ALL);
+ reg_write(dev, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
+
+ /* tclk_term_en */
+ reg_write_field(dev, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
+ /* tclk_settle */
+ reg_write_field(dev, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
+ /* td_term_en */
+ reg_write_field(dev, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
+ /* ths_settle */
+ reg_write_field(dev, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
+ /* trx_enable */
+ reg_write_field(dev, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
+
+ reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_SOE);
+
+ /* Packet compare setup - required to avoid missing frame ends */
+ val = 0;
+ set_field(&val, 1, UNICAM_PCE);
+ set_field(&val, 1, UNICAM_GI);
+ set_field(&val, 1, UNICAM_CPH);
+ set_field(&val, 0, UNICAM_PCVC_MASK);
+ set_field(&val, 1, UNICAM_PCDT_MASK);
+ reg_write(dev, UNICAM_CMP0, val);
+
+ /* Enable clock lane and set up terminations */
+ val = 0;
+ if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
+ /* CSI2 */
+ set_field(&val, 1, UNICAM_CLE);
+ set_field(&val, 1, UNICAM_CLLPE);
+ if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
+ set_field(&val, 1, UNICAM_CLTRE);
+ set_field(&val, 1, UNICAM_CLHSE);
+ }
+ } else {
+ /* CCP2 */
+ set_field(&val, 1, UNICAM_CLE);
+ set_field(&val, 1, UNICAM_CLHSE);
+ set_field(&val, 1, UNICAM_CLTRE);
+ }
+ reg_write(dev, UNICAM_CLK, val);
+
+ /*
+ * Enable required data lanes with appropriate terminations.
+ * The same value needs to be written to UNICAM_DATn registers for
+ * the active lanes, and 0 for inactive ones.
+ */
+ val = 0;
+ if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
+ /* CSI2 */
+ set_field(&val, 1, UNICAM_DLE);
+ set_field(&val, 1, UNICAM_DLLPE);
+ if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
+ set_field(&val, 1, UNICAM_DLTRE);
+ set_field(&val, 1, UNICAM_DLHSE);
+ }
+ } else {
+ /* CCP2 */
+ set_field(&val, 1, UNICAM_DLE);
+ set_field(&val, 1, UNICAM_DLHSE);
+ set_field(&val, 1, UNICAM_DLTRE);
+ }
+ reg_write(dev, UNICAM_DAT0, val);
+
+ if (dev->active_data_lanes == 1)
+ val = 0;
+ reg_write(dev, UNICAM_DAT1, val);
+
+ if (dev->max_data_lanes > 2) {
+ /*
+ * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
+ * instance supports more than 2 data lanes.
+ */
+ if (dev->active_data_lanes == 2)
+ val = 0;
+ reg_write(dev, UNICAM_DAT2, val);
+
+ if (dev->active_data_lanes == 3)
+ val = 0;
+ reg_write(dev, UNICAM_DAT3, val);
+ }
+
+ reg_write(dev, UNICAM_IBLS,
+ dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline);
+ size = dev->node[IMAGE_PAD].v_fmt.fmt.pix.sizeimage;
+ unicam_wr_dma_addr(dev, addr[IMAGE_PAD], size, IMAGE_PAD);
+ unicam_set_packing_config(dev);
+ unicam_cfg_image_id(dev);
+
+ val = reg_read(dev, UNICAM_MISC);
+ set_field(&val, 1, UNICAM_FL0);
+ set_field(&val, 1, UNICAM_FL1);
+ reg_write(dev, UNICAM_MISC, val);
+
+ if (dev->node[METADATA_PAD].streaming && dev->sensor_embedded_data) {
+ size = dev->node[METADATA_PAD].v_fmt.fmt.meta.buffersize;
+ unicam_enable_ed(dev);
+ unicam_wr_dma_addr(dev, addr[METADATA_PAD], size, METADATA_PAD);
+ }
+
+ /* Enable peripheral */
+ reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPE);
+
+ /* Load image pointers */
+ reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
+
+ /* Load embedded data buffer pointers if needed */
+ if (dev->node[METADATA_PAD].streaming && dev->sensor_embedded_data)
+ reg_write_field(dev, UNICAM_DCS, 1, UNICAM_LDP);
+
+ /*
+ * Enable trigger only for the first frame to
+ * sync correctly to the FS from the source.
+ */
+ reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_TFC);
+}
+
+static void unicam_disable(struct unicam_device *dev)
+{
+ /* Analogue lane control disable */
+ reg_write_field(dev, UNICAM_ANA, 1, UNICAM_DDL);
+
+ /* Stop the output engine */
+ reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_SOE);
+
+ /* Disable the data lanes. */
+ reg_write(dev, UNICAM_DAT0, 0);
+ reg_write(dev, UNICAM_DAT1, 0);
+
+ if (dev->max_data_lanes > 2) {
+ reg_write(dev, UNICAM_DAT2, 0);
+ reg_write(dev, UNICAM_DAT3, 0);
+ }
+
+ /* Peripheral reset */
+ reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
+ usleep_range(50, 100);
+ reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
+
+ /* Disable peripheral */
+ reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
+
+ /* Clear ED setup */
+ reg_write(dev, UNICAM_DCS, 0);
+
+ /* Disable all lane clocks */
+ clk_write(dev, 0);
+}
+
+static void unicam_return_buffers(struct unicam_node *node)
+{
+ struct unicam_buffer *buf, *tmp;
+ unsigned long flags;
+
+ spin_lock_irqsave(&node->dma_queue_lock, flags);
+ list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
+ list_del(&buf->list);
+ vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+
+ if (node->cur_frm)
+ vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
+ VB2_BUF_STATE_ERROR);
+ if (node->next_frm && node->cur_frm != node->next_frm)
+ vb2_buffer_done(&node->next_frm->vb.vb2_buf,
+ VB2_BUF_STATE_ERROR);
+
+ node->cur_frm = NULL;
+ node->next_frm = NULL;
+ spin_unlock_irqrestore(&node->dma_queue_lock, flags);
+}
+
+static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+ struct unicam_node *node = vb2_get_drv_priv(vq);
+ struct unicam_device *dev = node->dev;
+ dma_addr_t buffer_addr[MAX_NODES] = { 0 };
+ unsigned long flags;
+ unsigned int i;
+ int ret;
+
+ node->streaming = true;
+ if (!unicam_all_nodes_streaming(dev)) {
+ unicam_dbg(3, dev, "Not all nodes are streaming yet.");
+ return 0;
+ }
+
+ dev->sequence = 0;
+ ret = unicam_runtime_get(dev);
+ if (ret < 0) {
+ unicam_dbg(3, dev, "unicam_runtime_get failed\n");
+ goto err_streaming;
+ }
+
+ /*
+ * TODO: Retrieve the number of active data lanes from the connected
+ * subdevice.
+ */
+ dev->active_data_lanes = dev->max_data_lanes;
+
+ ret = clk_set_rate(dev->clock, 100 * 1000 * 1000);
+ if (ret) {
+ unicam_err(dev, "failed to set up clock\n");
+ goto err_pm_put;
+ }
+
+ ret = clk_prepare_enable(dev->clock);
+ if (ret) {
+ unicam_err(dev, "Failed to enable CSI clock: %d\n", ret);
+ goto err_pm_put;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(dev->node); i++) {
+ struct unicam_buffer *buf;
+
+ if (!dev->node[i].streaming)
+ continue;
+
+ spin_lock_irqsave(&dev->node[i].dma_queue_lock, flags);
+ buf = list_first_entry(&dev->node[i].dma_queue,
+ struct unicam_buffer, list);
+ dev->node[i].cur_frm = buf;
+ dev->node[i].next_frm = buf;
+ list_del(&buf->list);
+ spin_unlock_irqrestore(&dev->node[i].dma_queue_lock, flags);
+
+ buffer_addr[i] =
+ vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
+ }
+
+ unicam_start_rx(dev, buffer_addr);
+
+ ret = v4l2_subdev_call(dev->sensor, video, s_stream, 1);
+ if (ret < 0) {
+ unicam_err(dev, "stream on failed in subdev\n");
+ goto err_disable_unicam;
+ }
+
+ return 0;
+
+err_disable_unicam:
+ unicam_disable(dev);
+ clk_disable_unprepare(dev->clock);
+err_pm_put:
+ unicam_runtime_put(dev);
+err_streaming:
+ unicam_return_buffers(node);
+ node->streaming = false;
+
+ return ret;
+}
+
+static void unicam_stop_streaming(struct vb2_queue *vq)
+{
+ struct unicam_node *node = vb2_get_drv_priv(vq);
+ struct unicam_device *dev = node->dev;
+
+ node->streaming = false;
+
+ if (node->pad_id == IMAGE_PAD) {
+ /*
+ * Stop streaming the sensor and disable the peripheral.
+ * We cannot continue streaming embedded data with the
+ * image pad disabled.
+ */
+ if (v4l2_subdev_call(dev->sensor, video, s_stream, 0) < 0)
+ unicam_err(dev, "stream off failed in subdev\n");
+
+ unicam_disable(dev);
+ clk_disable_unprepare(dev->clock);
+ unicam_runtime_put(dev);
+
+ } else if (node->pad_id == METADATA_PAD) {
+ /*
+ * Allow the hardware to spin in the dummy buffer.
+ * This is only really needed if the embedded data pad is
+ * disabled before the image pad.
+ */
+ unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr,
+ DUMMY_BUF_SIZE, METADATA_PAD);
+ }
+
+ /* Clear all queued buffers for the node */
+ unicam_return_buffers(node);
+}
+
+static int unicam_enum_input(struct file *file, void *priv,
+ struct v4l2_input *inp)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ if (inp->index != 0)
+ return -EINVAL;
+
+ inp->type = V4L2_INPUT_TYPE_CAMERA;
+ if (v4l2_subdev_has_op(dev->sensor, video, s_dv_timings)) {
+ inp->capabilities = V4L2_IN_CAP_DV_TIMINGS;
+ inp->std = 0;
+ } else if (v4l2_subdev_has_op(dev->sensor, video, s_std)) {
+ inp->capabilities = V4L2_IN_CAP_STD;
+ if (v4l2_subdev_call(dev->sensor, video, g_tvnorms, &inp->std)
+ < 0)
+ inp->std = V4L2_STD_ALL;
+ } else {
+ inp->capabilities = 0;
+ inp->std = 0;
+ }
+ sprintf(inp->name, "Camera 0");
+ return 0;
+}
+
+static int unicam_g_input(struct file *file, void *priv, unsigned int *i)
+{
+ *i = 0;
+
+ return 0;
+}
+
+static int unicam_s_input(struct file *file, void *priv, unsigned int i)
+{
+ /*
+ * FIXME: Ideally we would like to be able to query the source
+ * subdevice for information over the input connectors it supports,
+ * and map that through in to a call to video_ops->s_routing.
+ * There is no infrastructure support for defining that within
+ * devicetree at present. Until that is implemented we can't
+ * map a user physical connector number to s_routing input number.
+ */
+ if (i > 0)
+ return -EINVAL;
+
+ return 0;
+}
+
+static int unicam_querystd(struct file *file, void *priv,
+ v4l2_std_id *std)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, video, querystd, std);
+}
+
+static int unicam_g_std(struct file *file, void *priv, v4l2_std_id *std)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, video, g_std, std);
+}
+
+static int unicam_s_std(struct file *file, void *priv, v4l2_std_id std)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ int ret;
+ v4l2_std_id current_std;
+
+ ret = v4l2_subdev_call(dev->sensor, video, g_std, ¤t_std);
+ if (ret)
+ return ret;
+
+ if (std == current_std)
+ return 0;
+
+ if (vb2_is_busy(&node->buffer_queue))
+ return -EBUSY;
+
+ ret = v4l2_subdev_call(dev->sensor, video, s_std, std);
+
+ /* Force recomputation of bytesperline */
+ node->v_fmt.fmt.pix.bytesperline = 0;
+
+ unicam_reset_format(node);
+
+ return ret;
+}
+
+static int unicam_s_edid(struct file *file, void *priv, struct v4l2_edid *edid)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, pad, set_edid, edid);
+}
+
+static int unicam_g_edid(struct file *file, void *priv, struct v4l2_edid *edid)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, pad, get_edid, edid);
+}
+
+static int unicam_s_selection(struct file *file, void *priv,
+ struct v4l2_selection *sel)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ struct v4l2_subdev_selection sdsel = {
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .target = sel->target,
+ .flags = sel->flags,
+ .r = sel->r,
+ };
+
+ return v4l2_subdev_call(dev->sensor, pad, set_selection, NULL, &sdsel);
+}
+
+static int unicam_g_selection(struct file *file, void *priv,
+ struct v4l2_selection *sel)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ struct v4l2_subdev_selection sdsel = {
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ .target = sel->target,
+ };
+ int ret;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, get_selection, NULL, &sdsel);
+ if (!ret)
+ sel->r = sdsel.r;
+
+ return ret;
+}
+
+static int unicam_enum_framesizes(struct file *file, void *priv,
+ struct v4l2_frmsizeenum *fsize)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ const struct unicam_fmt *fmt;
+ struct v4l2_subdev_frame_size_enum fse;
+ int ret;
+
+ /* check for valid format */
+ fmt = find_format_by_pix(dev, fsize->pixel_format);
+ if (!fmt) {
+ unicam_dbg(3, dev, "Invalid pixel code: %x\n",
+ fsize->pixel_format);
+ return -EINVAL;
+ }
+ fse.code = fmt->code;
+
+ fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+ fse.index = fsize->index;
+ fse.pad = node->pad_id;
+
+ ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_size, NULL, &fse);
+ if (ret)
+ return ret;
+
+ unicam_dbg(1, dev, "%s: index: %d code: %x W:[%d,%d] H:[%d,%d]\n",
+ __func__, fse.index, fse.code, fse.min_width, fse.max_width,
+ fse.min_height, fse.max_height);
+
+ fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+ fsize->discrete.width = fse.max_width;
+ fsize->discrete.height = fse.max_height;
+
+ return 0;
+}
+
+static int unicam_enum_frameintervals(struct file *file, void *priv,
+ struct v4l2_frmivalenum *fival)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ const struct unicam_fmt *fmt;
+ struct v4l2_subdev_frame_interval_enum fie = {
+ .index = fival->index,
+ .width = fival->width,
+ .height = fival->height,
+ .which = V4L2_SUBDEV_FORMAT_ACTIVE,
+ };
+ int ret;
+
+ fmt = find_format_by_pix(dev, fival->pixel_format);
+ if (!fmt)
+ return -EINVAL;
+
+ fie.code = fmt->code;
+ ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_interval,
+ NULL, &fie);
+ if (ret)
+ return ret;
+
+ fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
+ fival->discrete = fie.interval;
+
+ return 0;
+}
+
+static int unicam_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_g_parm_cap(video_devdata(file), dev->sensor, a);
+}
+
+static int unicam_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_s_parm_cap(video_devdata(file), dev->sensor, a);
+}
+
+static int unicam_g_dv_timings(struct file *file, void *priv,
+ struct v4l2_dv_timings *timings)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, video, g_dv_timings, timings);
+}
+
+static int unicam_s_dv_timings(struct file *file, void *priv,
+ struct v4l2_dv_timings *timings)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ struct v4l2_dv_timings current_timings;
+ int ret;
+
+ ret = v4l2_subdev_call(dev->sensor, video, g_dv_timings,
+ ¤t_timings);
+
+ if (v4l2_match_dv_timings(timings, ¤t_timings, 0, false))
+ return 0;
+
+ if (vb2_is_busy(&node->buffer_queue))
+ return -EBUSY;
+
+ ret = v4l2_subdev_call(dev->sensor, video, s_dv_timings, timings);
+
+ /* Force recomputation of bytesperline */
+ node->v_fmt.fmt.pix.bytesperline = 0;
+
+ unicam_reset_format(node);
+
+ return ret;
+}
+
+static int unicam_query_dv_timings(struct file *file, void *priv,
+ struct v4l2_dv_timings *timings)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, video, query_dv_timings, timings);
+}
+
+static int unicam_enum_dv_timings(struct file *file, void *priv,
+ struct v4l2_enum_dv_timings *timings)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, pad, enum_dv_timings, timings);
+}
+
+static int unicam_dv_timings_cap(struct file *file, void *priv,
+ struct v4l2_dv_timings_cap *cap)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+
+ return v4l2_subdev_call(dev->sensor, pad, dv_timings_cap, cap);
+}
+
+static int unicam_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *sub)
+{
+ switch (sub->type) {
+ case V4L2_EVENT_FRAME_SYNC:
+ return v4l2_event_subscribe(fh, sub, 2, NULL);
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return v4l2_event_subscribe(fh, sub, 4, NULL);
+ }
+
+ return v4l2_ctrl_subscribe_event(fh, sub);
+}
+
+static int unicam_log_status(struct file *file, void *fh)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ u32 reg;
+
+ /* status for sub devices */
+ v4l2_device_call_all(&dev->v4l2_dev, 0, core, log_status);
+
+ unicam_info(dev, "-----Receiver status-----\n");
+ unicam_info(dev, "V4L2 width/height: %ux%u\n",
+ node->v_fmt.fmt.pix.width, node->v_fmt.fmt.pix.height);
+ unicam_info(dev, "Mediabus format: %08x\n", node->fmt->code);
+ unicam_info(dev, "V4L2 format: %08x\n",
+ node->v_fmt.fmt.pix.pixelformat);
+ reg = reg_read(dev, UNICAM_IPIPE);
+ unicam_info(dev, "Unpacking/packing: %u / %u\n",
+ get_field(reg, UNICAM_PUM_MASK),
+ get_field(reg, UNICAM_PPM_MASK));
+ unicam_info(dev, "----Live data----\n");
+ unicam_info(dev, "Programmed stride: %4u\n",
+ reg_read(dev, UNICAM_IBLS));
+ unicam_info(dev, "Detected resolution: %ux%u\n",
+ reg_read(dev, UNICAM_IHSTA),
+ reg_read(dev, UNICAM_IVSTA));
+ unicam_info(dev, "Write pointer: %08x\n",
+ reg_read(dev, UNICAM_IBWP));
+
+ return 0;
+}
+
+static void unicam_notify(struct v4l2_subdev *sd,
+ unsigned int notification, void *arg)
+{
+ struct unicam_device *dev = to_unicam_device(sd->v4l2_dev);
+
+ switch (notification) {
+ case V4L2_DEVICE_NOTIFY_EVENT:
+ v4l2_event_queue(&dev->node[IMAGE_PAD].video_dev, arg);
+ break;
+ default:
+ break;
+ }
+}
+
+static const struct vb2_ops unicam_video_qops = {
+ .wait_prepare = vb2_ops_wait_prepare,
+ .wait_finish = vb2_ops_wait_finish,
+ .queue_setup = unicam_queue_setup,
+ .buf_prepare = unicam_buffer_prepare,
+ .buf_queue = unicam_buffer_queue,
+ .start_streaming = unicam_start_streaming,
+ .stop_streaming = unicam_stop_streaming,
+};
+
+/*
+ * unicam_v4l2_open : This function is based on the v4l2_fh_open helper
+ * function. It has been augmented to handle sensor subdevice power management,
+ */
+static int unicam_v4l2_open(struct file *file)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ int ret;
+
+ mutex_lock(&node->lock);
+
+ ret = v4l2_fh_open(file);
+ if (ret) {
+ unicam_err(dev, "v4l2_fh_open failed\n");
+ goto unlock;
+ }
+
+ node->open++;
+
+ if (!v4l2_fh_is_singular_file(file))
+ goto unlock;
+
+ ret = v4l2_subdev_call(dev->sensor, core, s_power, 1);
+ if (ret < 0 && ret != -ENOIOCTLCMD) {
+ v4l2_fh_release(file);
+ node->open--;
+ goto unlock;
+ }
+
+ ret = 0;
+
+unlock:
+ mutex_unlock(&node->lock);
+ return ret;
+}
+
+static int unicam_v4l2_release(struct file *file)
+{
+ struct unicam_node *node = video_drvdata(file);
+ struct unicam_device *dev = node->dev;
+ struct v4l2_subdev *sd = dev->sensor;
+ bool fh_singular;
+ int ret;
+
+ mutex_lock(&node->lock);
+
+ fh_singular = v4l2_fh_is_singular_file(file);
+
+ ret = _vb2_fop_release(file, NULL);
+
+ if (fh_singular)
+ v4l2_subdev_call(sd, core, s_power, 0);
+
+ node->open--;
+ mutex_unlock(&node->lock);
+
+ return ret;
+}
+
+/* unicam capture driver file operations */
+static const struct v4l2_file_operations unicam_fops = {
+ .owner = THIS_MODULE,
+ .open = unicam_v4l2_open,
+ .release = unicam_v4l2_release,
+ .read = vb2_fop_read,
+ .poll = vb2_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = vb2_fop_mmap,
+};
+
+/* unicam capture ioctl operations */
+static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
+ .vidioc_querycap = unicam_querycap,
+ .vidioc_enum_fmt_vid_cap = unicam_enum_fmt_vid_cap,
+ .vidioc_g_fmt_vid_cap = unicam_g_fmt_vid_cap,
+ .vidioc_s_fmt_vid_cap = unicam_s_fmt_vid_cap,
+ .vidioc_try_fmt_vid_cap = unicam_try_fmt_vid_cap,
+
+ .vidioc_enum_fmt_meta_cap = unicam_enum_fmt_meta_cap,
+ .vidioc_g_fmt_meta_cap = unicam_g_fmt_meta_cap,
+ .vidioc_s_fmt_meta_cap = unicam_g_fmt_meta_cap,
+ .vidioc_try_fmt_meta_cap = unicam_g_fmt_meta_cap,
+
+ .vidioc_enum_input = unicam_enum_input,
+ .vidioc_g_input = unicam_g_input,
+ .vidioc_s_input = unicam_s_input,
+
+ .vidioc_querystd = unicam_querystd,
+ .vidioc_s_std = unicam_s_std,
+ .vidioc_g_std = unicam_g_std,
+
+ .vidioc_g_edid = unicam_g_edid,
+ .vidioc_s_edid = unicam_s_edid,
+
+ .vidioc_enum_framesizes = unicam_enum_framesizes,
+ .vidioc_enum_frameintervals = unicam_enum_frameintervals,
+
+ .vidioc_g_selection = unicam_g_selection,
+ .vidioc_s_selection = unicam_s_selection,
+
+ .vidioc_g_parm = unicam_g_parm,
+ .vidioc_s_parm = unicam_s_parm,
+
+ .vidioc_s_dv_timings = unicam_s_dv_timings,
+ .vidioc_g_dv_timings = unicam_g_dv_timings,
+ .vidioc_query_dv_timings = unicam_query_dv_timings,
+ .vidioc_enum_dv_timings = unicam_enum_dv_timings,
+ .vidioc_dv_timings_cap = unicam_dv_timings_cap,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+
+ .vidioc_log_status = unicam_log_status,
+ .vidioc_subscribe_event = unicam_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+static int
+unicam_async_bound(struct v4l2_async_notifier *notifier,
+ struct v4l2_subdev *subdev,
+ struct v4l2_async_subdev *asd)
+{
+ struct unicam_device *unicam = to_unicam_device(notifier->v4l2_dev);
+
+ if (unicam->sensor) {
+ unicam_info(unicam, "Rejecting subdev %s (Already set!!)",
+ subdev->name);
+ return 0;
+ }
+
+ unicam->sensor = subdev;
+ unicam_dbg(1, unicam, "Using sensor %s for capture\n", subdev->name);
+
+ return 0;
+}
+
+static void unicam_release(struct kref *kref)
+{
+ struct unicam_device *unicam =
+ container_of(kref, struct unicam_device, kref);
+
+ v4l2_ctrl_handler_free(&unicam->ctrl_handler);
+ media_device_cleanup(&unicam->mdev);
+
+ if (unicam->sensor_config)
+ v4l2_subdev_free_pad_config(unicam->sensor_config);
+
+ kfree(unicam);
+}
+
+static void unicam_put(struct unicam_device *unicam)
+{
+ kref_put(&unicam->kref, unicam_release);
+}
+
+static void unicam_get(struct unicam_device *unicam)
+{
+ kref_get(&unicam->kref);
+}
+
+static void unicam_node_release(struct video_device *vdev)
+{
+ struct unicam_node *node = video_get_drvdata(vdev);
+
+ unicam_put(node->dev);
+}
+
+static int register_node(struct unicam_device *unicam, struct unicam_node *node,
+ enum v4l2_buf_type type, int pad_id)
+{
+ struct video_device *vdev;
+ struct vb2_queue *q;
+ struct v4l2_mbus_framefmt mbus_fmt = {0};
+ const struct unicam_fmt *fmt;
+ int ret;
+
+ if (pad_id == IMAGE_PAD) {
+ ret = __subdev_get_format(unicam, &mbus_fmt, pad_id);
+ if (ret) {
+ unicam_err(unicam, "Failed to get_format - ret %d\n",
+ ret);
+ return ret;
+ }
+
+ fmt = find_format_by_code(mbus_fmt.code);
+ if (!fmt) {
+ /*
+ * Find the first format that the sensor and unicam both
+ * support
+ */
+ fmt = get_first_supported_format(unicam);
+
+ if (!fmt)
+ /* No compatible formats */
+ return -EINVAL;
+
+ mbus_fmt.code = fmt->code;
+ ret = __subdev_set_format(unicam, &mbus_fmt, pad_id);
+ if (ret)
+ return -EINVAL;
+ }
+ if (mbus_fmt.field != V4L2_FIELD_NONE) {
+ /* Interlaced not supported - disable it now. */
+ mbus_fmt.field = V4L2_FIELD_NONE;
+ ret = __subdev_set_format(unicam, &mbus_fmt, pad_id);
+ if (ret)
+ return -EINVAL;
+ }
+
+ node->v_fmt.fmt.pix.pixelformat = fmt->fourcc ? fmt->fourcc
+ : fmt->repacked_fourcc;
+ } else {
+ /* Fix this node format as embedded data. */
+ fmt = find_format_by_code(MEDIA_BUS_FMT_SENSOR_DATA);
+ node->v_fmt.fmt.meta.dataformat = fmt->fourcc;
+ }
+
+ node->dev = unicam;
+ node->pad_id = pad_id;
+ node->fmt = fmt;
+
+ /* Read current subdev format */
+ unicam_reset_format(node);
+
+ if (v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
+ v4l2_std_id tvnorms;
+
+ if (WARN_ON(!v4l2_subdev_has_op(unicam->sensor, video,
+ g_tvnorms)))
+ /*
+ * Subdevice should not advertise s_std but not
+ * g_tvnorms
+ */
+ return -EINVAL;
+
+ ret = v4l2_subdev_call(unicam->sensor, video,
+ g_tvnorms, &tvnorms);
+ if (WARN_ON(ret))
+ return -EINVAL;
+ node->video_dev.tvnorms |= tvnorms;
+ }
+
+ spin_lock_init(&node->dma_queue_lock);
+ mutex_init(&node->lock);
+
+ vdev = &node->video_dev;
+ if (pad_id == IMAGE_PAD) {
+ /* Add controls from the subdevice */
+ ret = v4l2_ctrl_add_handler(&unicam->ctrl_handler,
+ unicam->sensor->ctrl_handler, NULL,
+ true);
+ if (ret < 0)
+ return ret;
+
+ /*
+ * If the sensor subdevice has any controls, associate the node
+ * with the ctrl handler to allow access from userland.
+ */
+ if (!list_empty(&unicam->ctrl_handler.ctrls))
+ vdev->ctrl_handler = &unicam->ctrl_handler;
+ }
+
+ q = &node->buffer_queue;
+ q->type = type;
+ q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;
+ q->drv_priv = node;
+ q->ops = &unicam_video_qops;
+ q->mem_ops = &vb2_dma_contig_memops;
+ q->buf_struct_size = sizeof(struct unicam_buffer);
+ q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
+ q->lock = &node->lock;
+ q->min_buffers_needed = 2;
+ q->dev = &unicam->pdev->dev;
+
+ ret = vb2_queue_init(q);
+ if (ret) {
+ unicam_err(unicam, "vb2_queue_init() failed\n");
+ return ret;
+ }
+
+ INIT_LIST_HEAD(&node->dma_queue);
+
+ vdev->release = unicam_node_release;
+ vdev->fops = &unicam_fops;
+ vdev->ioctl_ops = &unicam_ioctl_ops;
+ vdev->v4l2_dev = &unicam->v4l2_dev;
+ vdev->vfl_dir = VFL_DIR_RX;
+ vdev->queue = q;
+ vdev->lock = &node->lock;
+ vdev->device_caps = (pad_id == IMAGE_PAD) ?
+ (V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING) :
+ (V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING);
+
+ /* Define the device names */
+ snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
+ pad_id == IMAGE_PAD ? "image" : "embedded");
+
+ video_set_drvdata(vdev, node);
+ if (pad_id == IMAGE_PAD)
+ vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
+ node->pad.flags = MEDIA_PAD_FL_SINK;
+ media_entity_pads_init(&vdev->entity, 1, &node->pad);
+
+ node->dummy_buf_cpu_addr = dma_alloc_coherent(&unicam->pdev->dev,
+ DUMMY_BUF_SIZE,
+ &node->dummy_buf_dma_addr,
+ GFP_KERNEL);
+ if (!node->dummy_buf_cpu_addr) {
+ unicam_err(unicam, "Unable to allocate dummy buffer.\n");
+ return -ENOMEM;
+ }
+
+ if (pad_id == METADATA_PAD) {
+ v4l2_disable_ioctl(vdev, VIDIOC_DQEVENT);
+ v4l2_disable_ioctl(vdev, VIDIOC_SUBSCRIBE_EVENT);
+ v4l2_disable_ioctl(vdev, VIDIOC_UNSUBSCRIBE_EVENT);
+ }
+ if (pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_STD);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_STD);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUMSTD);
+ }
+ if (pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, video, querystd))
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_QUERYSTD);
+ if (pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, video, s_dv_timings)) {
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_EDID);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_EDID);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_DV_TIMINGS_CAP);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_DV_TIMINGS);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_DV_TIMINGS);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUM_DV_TIMINGS);
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_QUERY_DV_TIMINGS);
+ }
+ if (pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, pad, enum_frame_interval))
+ v4l2_disable_ioctl(&node->video_dev,
+ VIDIOC_ENUM_FRAMEINTERVALS);
+ if (pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, video, g_frame_interval))
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_PARM);
+ if (pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, video, s_frame_interval))
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_PARM);
+
+ if (pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, pad, enum_frame_size))
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUM_FRAMESIZES);
+
+ if (node->pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, pad, set_selection))
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_SELECTION);
+
+ if (node->pad_id == METADATA_PAD ||
+ !v4l2_subdev_has_op(unicam->sensor, pad, get_selection))
+ v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_SELECTION);
+
+ ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
+ if (ret) {
+ unicam_err(unicam, "Unable to register video device %s\n",
+ vdev->name);
+ return ret;
+ }
+
+ /*
+ * Acquire a reference to unicam, which will be released when the video
+ * device will be unregistered and userspace will have closed all open
+ * file handles.
+ */
+ unicam_get(unicam);
+ node->registered = true;
+
+ if (pad_id != METADATA_PAD || unicam->sensor_embedded_data) {
+ ret = media_create_pad_link(&unicam->sensor->entity, pad_id,
+ &node->video_dev.entity, 0,
+ MEDIA_LNK_FL_ENABLED |
+ MEDIA_LNK_FL_IMMUTABLE);
+ if (ret)
+ unicam_err(unicam, "Unable to create pad link for %s\n",
+ vdev->name);
+ }
+
+ return ret;
+}
+
+static void unregister_nodes(struct unicam_device *unicam)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
+ struct unicam_node *node = &unicam->node[i];
+
+ if (node->dummy_buf_cpu_addr) {
+ dma_free_coherent(&unicam->pdev->dev, DUMMY_BUF_SIZE,
+ node->dummy_buf_cpu_addr,
+ node->dummy_buf_dma_addr);
+ }
+
+ if (node->registered) {
+ node->registered = false;
+ video_unregister_device(&node->video_dev);
+ }
+ }
+}
+
+static int unicam_probe_complete(struct unicam_device *unicam)
+{
+ int ret;
+
+ unicam->v4l2_dev.notify = unicam_notify;
+
+ unicam->sensor_config = v4l2_subdev_alloc_pad_config(unicam->sensor);
+ if (!unicam->sensor_config)
+ return -ENOMEM;
+
+ unicam->sensor_embedded_data = (unicam->sensor->entity.num_pads >= 2);
+
+ ret = register_node(unicam, &unicam->node[IMAGE_PAD],
+ V4L2_BUF_TYPE_VIDEO_CAPTURE, IMAGE_PAD);
+ if (ret) {
+ unicam_err(unicam, "Unable to register image video device.\n");
+ goto unregister;
+ }
+
+ ret = register_node(unicam, &unicam->node[METADATA_PAD],
+ V4L2_BUF_TYPE_META_CAPTURE, METADATA_PAD);
+ if (ret) {
+ unicam_err(unicam, "Unable to register metadata video device.\n");
+ goto unregister;
+ }
+
+ ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
+ if (ret) {
+ unicam_err(unicam, "Unable to register subdev nodes.\n");
+ goto unregister;
+ }
+
+ /*
+ * Release the initial reference, all references are now owned by the
+ * video devices.
+ */
+ unicam_put(unicam);
+ return 0;
+
+unregister:
+ unregister_nodes(unicam);
+ unicam_put(unicam);
+
+ return ret;
+}
+
+static int unicam_async_complete(struct v4l2_async_notifier *notifier)
+{
+ struct unicam_device *unicam = to_unicam_device(notifier->v4l2_dev);
+
+ return unicam_probe_complete(unicam);
+}
+
+static const struct v4l2_async_notifier_operations unicam_async_ops = {
+ .bound = unicam_async_bound,
+ .complete = unicam_async_complete,
+};
+
+static int of_unicam_connect_subdevs(struct unicam_device *dev)
+{
+ struct platform_device *pdev = dev->pdev;
+ struct v4l2_fwnode_endpoint ep = { 0 };
+ struct device_node *ep_node;
+ struct device_node *sensor_node;
+ unsigned int lane;
+ int ret = -EINVAL;
+
+ if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
+ &dev->max_data_lanes) < 0) {
+ unicam_err(dev, "number of data lanes not set\n");
+ return -EINVAL;
+ }
+
+ /* Get the local endpoint and remote device. */
+ ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
+ if (!ep_node) {
+ unicam_dbg(3, dev, "can't get next endpoint\n");
+ return -EINVAL;
+ }
+
+ unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
+
+ sensor_node = of_graph_get_remote_port_parent(ep_node);
+ if (!sensor_node) {
+ unicam_dbg(3, dev, "can't get remote parent\n");
+ goto cleanup_exit;
+ }
+
+ unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
+
+ /* Parse the local endpoint and validate its configuration. */
+ v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
+
+ unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
+ ep.bus_type);
+
+ dev->bus_type = ep.bus_type;
+
+ switch (ep.bus_type) {
+ case V4L2_MBUS_CSI2_DPHY:
+ switch (ep.bus.mipi_csi2.num_data_lanes) {
+ case 1:
+ case 2:
+ case 4:
+ break;
+
+ default:
+ unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
+ sensor_node,
+ ep.bus.mipi_csi2.num_data_lanes);
+ goto cleanup_exit;
+ }
+
+ for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
+ if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
+ unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
+ sensor_node);
+ goto cleanup_exit;
+ }
+ }
+
+ if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
+ unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
+ ep.bus.mipi_csi2.num_data_lanes,
+ dev->max_data_lanes);
+ }
+
+ dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
+ dev->bus_flags = ep.bus.mipi_csi2.flags;
+
+ break;
+
+ case V4L2_MBUS_CCP2:
+ if (ep.bus.mipi_csi1.clock_lane != 0 ||
+ ep.bus.mipi_csi1.data_lane != 1) {
+ unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
+ sensor_node);
+ goto cleanup_exit;
+ }
+
+ dev->max_data_lanes = 1;
+ dev->bus_flags = ep.bus.mipi_csi1.strobe;
+ break;
+
+ default:
+ /* Unsupported bus type */
+ unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
+ sensor_node, ep.bus_type);
+ goto cleanup_exit;
+ }
+
+ unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
+ sensor_node,
+ dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
+ dev->max_data_lanes, dev->bus_flags);
+
+ /* Initialize and register the async notifier. */
+ v4l2_async_notifier_init(&dev->notifier);
+ dev->notifier.ops = &unicam_async_ops;
+
+ dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
+ dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
+ ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
+ if (ret) {
+ unicam_err(dev, "Error adding subdevice: %d\n", ret);
+ goto cleanup_exit;
+ }
+
+ ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
+ if (ret) {
+ unicam_err(dev, "Error registering async notifier: %d\n", ret);
+ ret = -EINVAL;
+ }
+
+cleanup_exit:
+ of_node_put(sensor_node);
+ of_node_put(ep_node);
+
+ return ret;
+}
+
+static int unicam_probe(struct platform_device *pdev)
+{
+ struct unicam_device *unicam;
+ int ret;
+
+ unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
+ if (!unicam)
+ return -ENOMEM;
+
+ kref_init(&unicam->kref);
+ unicam->pdev = pdev;
+
+ unicam->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(unicam->base)) {
+ unicam_err(unicam, "Failed to get main io block\n");
+ ret = PTR_ERR(unicam->base);
+ goto err_unicam_put;
+ }
+
+ unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
+ if (IS_ERR(unicam->clk_gate_base)) {
+ unicam_err(unicam, "Failed to get 2nd io block\n");
+ ret = PTR_ERR(unicam->clk_gate_base);
+ goto err_unicam_put;
+ }
+
+ unicam->clock = devm_clk_get(&pdev->dev, "lp");
+ if (IS_ERR(unicam->clock)) {
+ unicam_err(unicam, "Failed to get clock\n");
+ ret = PTR_ERR(unicam->clock);
+ goto err_unicam_put;
+ }
+
+ ret = platform_get_irq(pdev, 0);
+ if (ret <= 0) {
+ dev_err(&pdev->dev, "No IRQ resource\n");
+ ret = -EINVAL;
+ goto err_unicam_put;
+ }
+
+ ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
+ "unicam_capture0", unicam);
+ if (ret) {
+ dev_err(&pdev->dev, "Unable to request interrupt\n");
+ ret = -EINVAL;
+ goto err_unicam_put;
+ }
+
+ unicam->mdev.dev = &pdev->dev;
+ strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
+ sizeof(unicam->mdev.model));
+ strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
+ snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
+ "platform:%s", dev_name(&pdev->dev));
+ unicam->mdev.hw_revision = 0;
+
+ media_device_init(&unicam->mdev);
+
+ unicam->v4l2_dev.mdev = &unicam->mdev;
+
+ ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
+ if (ret) {
+ unicam_err(unicam,
+ "Unable to register v4l2 device.\n");
+ goto err_unicam_put;
+ }
+
+ ret = media_device_register(&unicam->mdev);
+ if (ret < 0) {
+ unicam_err(unicam,
+ "Unable to register media-controller device.\n");
+ goto err_v4l2_unregister;
+ }
+
+ /* Reserve space for the controls */
+ ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
+ if (ret < 0)
+ goto err_media_unregister;
+
+ /* set the driver data in platform device */
+ platform_set_drvdata(pdev, unicam);
+
+ ret = of_unicam_connect_subdevs(unicam);
+ if (ret) {
+ dev_err(&pdev->dev, "Failed to connect subdevs\n");
+ goto err_media_unregister;
+ }
+
+ /* Enable the block power domain */
+ pm_runtime_enable(&pdev->dev);
+
+ return 0;
+
+err_media_unregister:
+ media_device_unregister(&unicam->mdev);
+err_v4l2_unregister:
+ v4l2_device_unregister(&unicam->v4l2_dev);
+err_unicam_put:
+ unicam_put(unicam);
+
+ return ret;
+}
+
+static int unicam_remove(struct platform_device *pdev)
+{
+ struct unicam_device *unicam = platform_get_drvdata(pdev);
+
+ unicam_dbg(2, unicam, "%s\n", __func__);
+
+ v4l2_async_notifier_unregister(&unicam->notifier);
+ v4l2_device_unregister(&unicam->v4l2_dev);
+ media_device_unregister(&unicam->mdev);
+ unregister_nodes(unicam);
+
+ pm_runtime_disable(&pdev->dev);
+
+ return 0;
+}
+
+static const struct of_device_id unicam_of_match[] = {
+ { .compatible = "brcm,bcm2835-unicam", },
+ { /* sentinel */ },
+};
+MODULE_DEVICE_TABLE(of, unicam_of_match);
+
+static struct platform_driver unicam_driver = {
+ .probe = unicam_probe,
+ .remove = unicam_remove,
+ .driver = {
+ .name = UNICAM_MODULE_NAME,
+ .of_match_table = of_match_ptr(unicam_of_match),
+ },
+};
+
+module_platform_driver(unicam_driver);
+
+MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
+MODULE_DESCRIPTION("BCM2835 Unicam driver");
+MODULE_LICENSE("GPL");
+MODULE_VERSION(UNICAM_VERSION);
diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
new file mode 100644
index 000000000000..ae059a171d0f
--- /dev/null
+++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
@@ -0,0 +1,253 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+
+/*
+ * Copyright (C) 2017-2020 Raspberry Pi Trading.
+ * Dave Stevenson <dave.stevenson@raspberrypi.com>
+ */
+
+#ifndef VC4_REGS_UNICAM_H
+#define VC4_REGS_UNICAM_H
+
+/*
+ * The following values are taken from files found within the code drop
+ * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
+ * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
+ * They have been modified to be only the register offset.
+ */
+#define UNICAM_CTRL 0x000
+#define UNICAM_STA 0x004
+#define UNICAM_ANA 0x008
+#define UNICAM_PRI 0x00c
+#define UNICAM_CLK 0x010
+#define UNICAM_CLT 0x014
+#define UNICAM_DAT0 0x018
+#define UNICAM_DAT1 0x01c
+#define UNICAM_DAT2 0x020
+#define UNICAM_DAT3 0x024
+#define UNICAM_DLT 0x028
+#define UNICAM_CMP0 0x02c
+#define UNICAM_CMP1 0x030
+#define UNICAM_CAP0 0x034
+#define UNICAM_CAP1 0x038
+#define UNICAM_ICTL 0x100
+#define UNICAM_ISTA 0x104
+#define UNICAM_IDI0 0x108
+#define UNICAM_IPIPE 0x10c
+#define UNICAM_IBSA0 0x110
+#define UNICAM_IBEA0 0x114
+#define UNICAM_IBLS 0x118
+#define UNICAM_IBWP 0x11c
+#define UNICAM_IHWIN 0x120
+#define UNICAM_IHSTA 0x124
+#define UNICAM_IVWIN 0x128
+#define UNICAM_IVSTA 0x12c
+#define UNICAM_ICC 0x130
+#define UNICAM_ICS 0x134
+#define UNICAM_IDC 0x138
+#define UNICAM_IDPO 0x13c
+#define UNICAM_IDCA 0x140
+#define UNICAM_IDCD 0x144
+#define UNICAM_IDS 0x148
+#define UNICAM_DCS 0x200
+#define UNICAM_DBSA0 0x204
+#define UNICAM_DBEA0 0x208
+#define UNICAM_DBWP 0x20c
+#define UNICAM_DBCTL 0x300
+#define UNICAM_IBSA1 0x304
+#define UNICAM_IBEA1 0x308
+#define UNICAM_IDI1 0x30c
+#define UNICAM_DBSA1 0x310
+#define UNICAM_DBEA1 0x314
+#define UNICAM_MISC 0x400
+
+/*
+ * The following bitmasks are from the kernel released by Broadcom
+ * for Android - https://android.googlesource.com/kernel/bcm/
+ * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
+ * Unicam block as BCM2835, as defined in eg
+ * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
+ * Values reworked to use the kernel BIT and GENMASK macros.
+ *
+ * Some of the bit mnenomics have been amended to match the datasheet.
+ */
+/* UNICAM_CTRL Register */
+#define UNICAM_CPE BIT(0)
+#define UNICAM_MEM BIT(1)
+#define UNICAM_CPR BIT(2)
+#define UNICAM_CPM_MASK GENMASK(3, 3)
+#define UNICAM_CPM_CSI2 0
+#define UNICAM_CPM_CCP2 1
+#define UNICAM_SOE BIT(4)
+#define UNICAM_DCM_MASK GENMASK(5, 5)
+#define UNICAM_DCM_STROBE 0
+#define UNICAM_DCM_DATA 1
+#define UNICAM_SLS BIT(6)
+#define UNICAM_PFT_MASK GENMASK(11, 8)
+#define UNICAM_OET_MASK GENMASK(20, 12)
+
+/* UNICAM_STA Register */
+#define UNICAM_SYN BIT(0)
+#define UNICAM_CS BIT(1)
+#define UNICAM_SBE BIT(2)
+#define UNICAM_PBE BIT(3)
+#define UNICAM_HOE BIT(4)
+#define UNICAM_PLE BIT(5)
+#define UNICAM_SSC BIT(6)
+#define UNICAM_CRCE BIT(7)
+#define UNICAM_OES BIT(8)
+#define UNICAM_IFO BIT(9)
+#define UNICAM_OFO BIT(10)
+#define UNICAM_BFO BIT(11)
+#define UNICAM_DL BIT(12)
+#define UNICAM_PS BIT(13)
+#define UNICAM_IS BIT(14)
+#define UNICAM_PI0 BIT(15)
+#define UNICAM_PI1 BIT(16)
+#define UNICAM_FSI_S BIT(17)
+#define UNICAM_FEI_S BIT(18)
+#define UNICAM_LCI_S BIT(19)
+#define UNICAM_BUF0_RDY BIT(20)
+#define UNICAM_BUF0_NO BIT(21)
+#define UNICAM_BUF1_RDY BIT(22)
+#define UNICAM_BUF1_NO BIT(23)
+#define UNICAM_DI BIT(24)
+
+#define UNICAM_STA_MASK_ALL \
+ (UNICAM_DL + \
+ UNICAM_SBE + \
+ UNICAM_PBE + \
+ UNICAM_HOE + \
+ UNICAM_PLE + \
+ UNICAM_SSC + \
+ UNICAM_CRCE + \
+ UNICAM_IFO + \
+ UNICAM_OFO + \
+ UNICAM_PS + \
+ UNICAM_PI0 + \
+ UNICAM_PI1)
+
+/* UNICAM_ANA Register */
+#define UNICAM_APD BIT(0)
+#define UNICAM_BPD BIT(1)
+#define UNICAM_AR BIT(2)
+#define UNICAM_DDL BIT(3)
+#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
+#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
+
+/* UNICAM_PRI Register */
+#define UNICAM_PE BIT(0)
+#define UNICAM_PT_MASK GENMASK(2, 1)
+#define UNICAM_NP_MASK GENMASK(7, 4)
+#define UNICAM_PP_MASK GENMASK(11, 8)
+#define UNICAM_BS_MASK GENMASK(15, 12)
+#define UNICAM_BL_MASK GENMASK(17, 16)
+
+/* UNICAM_CLK Register */
+#define UNICAM_CLE BIT(0)
+#define UNICAM_CLPD BIT(1)
+#define UNICAM_CLLPE BIT(2)
+#define UNICAM_CLHSE BIT(3)
+#define UNICAM_CLTRE BIT(4)
+#define UNICAM_CLAC_MASK GENMASK(8, 5)
+#define UNICAM_CLSTE BIT(29)
+
+/* UNICAM_CLT Register */
+#define UNICAM_CLT1_MASK GENMASK(7, 0)
+#define UNICAM_CLT2_MASK GENMASK(15, 8)
+
+/* UNICAM_DATn Registers */
+#define UNICAM_DLE BIT(0)
+#define UNICAM_DLPD BIT(1)
+#define UNICAM_DLLPE BIT(2)
+#define UNICAM_DLHSE BIT(3)
+#define UNICAM_DLTRE BIT(4)
+#define UNICAM_DLSM BIT(5)
+#define UNICAM_DLFO BIT(28)
+#define UNICAM_DLSTE BIT(29)
+
+#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
+
+/* UNICAM_DLT Register */
+#define UNICAM_DLT1_MASK GENMASK(7, 0)
+#define UNICAM_DLT2_MASK GENMASK(15, 8)
+#define UNICAM_DLT3_MASK GENMASK(23, 16)
+
+/* UNICAM_ICTL Register */
+#define UNICAM_FSIE BIT(0)
+#define UNICAM_FEIE BIT(1)
+#define UNICAM_IBOB BIT(2)
+#define UNICAM_FCM BIT(3)
+#define UNICAM_TFC BIT(4)
+#define UNICAM_LIP_MASK GENMASK(6, 5)
+#define UNICAM_LCIE_MASK GENMASK(28, 16)
+
+/* UNICAM_IDI0/1 Register */
+#define UNICAM_ID0_MASK GENMASK(7, 0)
+#define UNICAM_ID1_MASK GENMASK(15, 8)
+#define UNICAM_ID2_MASK GENMASK(23, 16)
+#define UNICAM_ID3_MASK GENMASK(31, 24)
+
+/* UNICAM_ISTA Register */
+#define UNICAM_FSI BIT(0)
+#define UNICAM_FEI BIT(1)
+#define UNICAM_LCI BIT(2)
+
+#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
+
+/* UNICAM_IPIPE Register */
+#define UNICAM_PUM_MASK GENMASK(2, 0)
+ /* Unpacking modes */
+ #define UNICAM_PUM_NONE 0
+ #define UNICAM_PUM_UNPACK6 1
+ #define UNICAM_PUM_UNPACK7 2
+ #define UNICAM_PUM_UNPACK8 3
+ #define UNICAM_PUM_UNPACK10 4
+ #define UNICAM_PUM_UNPACK12 5
+ #define UNICAM_PUM_UNPACK14 6
+ #define UNICAM_PUM_UNPACK16 7
+#define UNICAM_DDM_MASK GENMASK(6, 3)
+#define UNICAM_PPM_MASK GENMASK(9, 7)
+ /* Packing modes */
+ #define UNICAM_PPM_NONE 0
+ #define UNICAM_PPM_PACK8 1
+ #define UNICAM_PPM_PACK10 2
+ #define UNICAM_PPM_PACK12 3
+ #define UNICAM_PPM_PACK14 4
+ #define UNICAM_PPM_PACK16 5
+#define UNICAM_DEM_MASK GENMASK(11, 10)
+#define UNICAM_DEBL_MASK GENMASK(14, 12)
+#define UNICAM_ICM_MASK GENMASK(16, 15)
+#define UNICAM_IDM_MASK GENMASK(17, 17)
+
+/* UNICAM_ICC Register */
+#define UNICAM_ICFL_MASK GENMASK(4, 0)
+#define UNICAM_ICFH_MASK GENMASK(9, 5)
+#define UNICAM_ICST_MASK GENMASK(12, 10)
+#define UNICAM_ICLT_MASK GENMASK(15, 13)
+#define UNICAM_ICLL_MASK GENMASK(31, 16)
+
+/* UNICAM_DCS Register */
+#define UNICAM_DIE BIT(0)
+#define UNICAM_DIM BIT(1)
+#define UNICAM_DBOB BIT(3)
+#define UNICAM_FDE BIT(4)
+#define UNICAM_LDP BIT(5)
+#define UNICAM_EDL_MASK GENMASK(15, 8)
+
+/* UNICAM_DBCTL Register */
+#define UNICAM_DBEN BIT(0)
+#define UNICAM_BUF0_IE BIT(1)
+#define UNICAM_BUF1_IE BIT(2)
+
+/* UNICAM_CMP[0,1] register */
+#define UNICAM_PCE BIT(31)
+#define UNICAM_GI BIT(9)
+#define UNICAM_CPH BIT(8)
+#define UNICAM_PCVC_MASK GENMASK(7, 6)
+#define UNICAM_PCDT_MASK GENMASK(5, 0)
+
+/* UNICAM_MISC register */
+#define UNICAM_FL0 BIT(6)
+#define UNICAM_FL1 BIT(9)
+
+#endif
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-05-04 9:25 ` [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface Laurent Pinchart
@ 2020-05-04 15:21 ` Hans Verkuil
2020-05-05 1:26 ` kbuild test robot
` (3 subsequent siblings)
4 siblings, 0 replies; 111+ messages in thread
From: Hans Verkuil @ 2020-05-04 15:21 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Some quick review comments:
On 04/05/2020 11:25, Laurent Pinchart wrote:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Add a driver for the Unicam camera receiver block on BCM283x processors.
> Compared to the bcm2835-camera driver present in staging, this driver
> handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> the VC4 firmware running on the VPU.
>
> The commit is made up of a series of changes cherry-picked from the
> rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> additional enhancements, forward-ported to the mainline kernel.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v1:
>
> - Re-fetch mbus code from subdev on a g_fmt call
> - Group all ioctl disabling together
> - Fix reference counting in unicam_open
> - Add support for VIDIOC_[S|G]_SELECTION
> ---
> MAINTAINERS | 7 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/bcm2835/Kconfig | 15 +
> drivers/media/platform/bcm2835/Makefile | 3 +
> .../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
> .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> 7 files changed, 3106 insertions(+)
> create mode 100644 drivers/media/platform/bcm2835/Kconfig
> create mode 100644 drivers/media/platform/bcm2835/Makefile
> create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b816a453b10e..edf5b8d9c98a 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3341,6 +3341,13 @@ N: bcm113*
> N: bcm216*
> N: kona
>
> +BROADCOM BCM2835 CAMERA DRIVER
> +M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> +L: linux-media@vger.kernel.org
> +S: Maintained
> +F: drivers/media/platform/bcm2835/
> +F: Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> +
> BROADCOM BCM47XX MIPS ARCHITECTURE
> M: Hauke Mehrtens <hauke@hauke-m.de>
> M: Rafał Miłecki <zajec5@gmail.com>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index e01bbb9dd1c1..98721a4e0be1 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -146,6 +146,7 @@ source "drivers/media/platform/am437x/Kconfig"
> source "drivers/media/platform/xilinx/Kconfig"
> source "drivers/media/platform/rcar-vin/Kconfig"
> source "drivers/media/platform/atmel/Kconfig"
> +source "drivers/media/platform/bcm2835/Kconfig"
> source "drivers/media/platform/sunxi/Kconfig"
>
> config VIDEO_TI_CAL
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index d13db96e3015..a425e4d2e3f3 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -98,4 +98,6 @@ obj-y += meson/
>
> obj-y += cros-ec-cec/
>
> +obj-y += bcm2835/
> +
> obj-y += sunxi/
> diff --git a/drivers/media/platform/bcm2835/Kconfig b/drivers/media/platform/bcm2835/Kconfig
> new file mode 100644
> index 000000000000..ec46e3ef053c
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/Kconfig
> @@ -0,0 +1,15 @@
> +# Broadcom VideoCore4 V4L2 camera support
> +
> +config VIDEO_BCM2835_UNICAM
> + tristate "Broadcom BCM2835 Unicam video capture driver"
> + depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
> + depends on ARCH_BCM2835 || COMPILE_TEST
> + select VIDEOBUF2_DMA_CONTIG
> + select V4L2_FWNODE
> + help
> + Say Y here to enable support for the BCM2835 CSI-2 receiver. This is a
> + V4L2 driver that controls the CSI-2 receiver directly, independently
> + from the VC4 firmware.
> +
> + To compile this driver as a module, choose M here. The module will be
> + called bcm2835-unicam.
> diff --git a/drivers/media/platform/bcm2835/Makefile b/drivers/media/platform/bcm2835/Makefile
> new file mode 100644
> index 000000000000..a98aba03598a
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/Makefile
> @@ -0,0 +1,3 @@
> +# Makefile for BCM2835 Unicam driver
> +
> +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> new file mode 100644
> index 000000000000..2e9387cbc1e0
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> @@ -0,0 +1,2825 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * BCM2835 Unicam Capture Driver
> + *
> + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> + *
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + *
> + * Based on TI am437x driver by
> + * Benoit Parrot <bparrot@ti.com>
> + * Lad, Prabhakar <prabhakar.csengg@gmail.com>
> + *
> + * and TI CAL camera interface driver by
> + * Benoit Parrot <bparrot@ti.com>
> + *
> + *
> + * There are two camera drivers in the kernel for BCM283x - this one
> + * and bcm2835-camera (currently in staging).
> + *
> + * This driver directly controls the Unicam peripheral - there is no
> + * involvement with the VideoCore firmware. Unicam receives CSI-2 or
> + * CCP2 data and writes it into SDRAM.
> + * The only potential processing options are to repack Bayer data into an
> + * alternate format, and applying windowing.
> + * The repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P
> + * to V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12,
> + * but not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both
> + * formats where the relevant formats are defined, and will automatically
> + * configure the repacking as required.
> + * Support for windowing may be added later.
> + *
> + * It should be possible to connect this driver to any sensor with a
> + * suitable output interface and V4L2 subdevice driver.
> + *
> + * bcm2835-camera uses the VideoCore firmware to control the sensor,
> + * Unicam, ISP, and all tuner control loops. Fully processed frames are
> + * delivered to the driver by the firmware. It only has sensor drivers
> + * for Omnivision OV5647, and Sony IMX219 sensors.
> + *
> + * The two drivers are mutually exclusive for the same Unicam instance.
> + * The VideoCore firmware checks the device tree configuration during boot.
> + * If it finds device tree nodes called csi0 or csi1 it will block the
> + * firmware from accessing the peripheral, and bcm2835-camera will
> + * not be able to stream data.
Very useful information. Should perhaps be added to the Kconfig description
as well.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/init.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/of_graph.h>
> +#include <linux/pinctrl/consumer.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> +#include <linux/uaccess.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-dv-timings.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "vc4-regs-unicam.h"
> +
> +#define UNICAM_MODULE_NAME "unicam"
> +#define UNICAM_VERSION "0.1.0"
> +
> +static int debug;
> +module_param(debug, int, 0644);
> +MODULE_PARM_DESC(debug, "Debug level 0-3");
> +
> +#define unicam_dbg(level, dev, fmt, arg...) \
> + v4l2_dbg(level, debug, &(dev)->v4l2_dev, fmt, ##arg)
> +#define unicam_info(dev, fmt, arg...) \
> + v4l2_info(&(dev)->v4l2_dev, fmt, ##arg)
> +#define unicam_err(dev, fmt, arg...) \
> + v4l2_err(&(dev)->v4l2_dev, fmt, ##arg)
> +
> +/*
> + * To protect against a dodgy sensor driver never returning an error from
> + * enum_mbus_code, set a maximum index value to be used.
> + */
> +#define MAX_ENUM_MBUS_CODE 128
> +
> +/*
> + * Stride is a 16 bit register, but also has to be a multiple of 32.
> + */
> +#define BPL_ALIGNMENT 32
> +#define MAX_BYTESPERLINE ((1 << 16) - BPL_ALIGNMENT)
> +/*
> + * Max width is therefore determined by the max stride divided by
> + * the number of bits per pixel. Take 32bpp as a
> + * worst case.
> + * No imposed limit on the height, so adopt a square image for want
> + * of anything better.
> + */
> +#define MAX_WIDTH (MAX_BYTESPERLINE / 4)
> +#define MAX_HEIGHT MAX_WIDTH
> +/* Define a nominal minimum image size */
> +#define MIN_WIDTH 16
> +#define MIN_HEIGHT 16
> +/* Default size of the embedded buffer */
> +#define UNICAM_EMBEDDED_SIZE 8192
> +
> +/*
> + * Size of the dummy buffer. Can be any size really, but the DMA
> + * allocation works in units of page sizes.
> + */
> +#define DUMMY_BUF_SIZE (PAGE_SIZE)
> +
> +enum pad_types {
> + IMAGE_PAD,
> + METADATA_PAD,
> + MAX_NODES
> +};
> +
> +/*
> + * struct unicam_fmt - Unicam media bus format information
> + * @pixelformat: V4L2 pixel format FCC identifier. 0 if n/a.
> + * @repacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> + * out to 16bpp. 0 if n/a.
> + * @code: V4L2 media bus format code.
> + * @depth: Bits per pixel as delivered from the source.
> + * @csi_dt: CSI data type.
> + * @check_variants: Flag to denote that there are multiple mediabus formats
> + * still in the list that could match this V4L2 format.
> + */
> +struct unicam_fmt {
> + u32 fourcc;
> + u32 repacked_fourcc;
> + u32 code;
> + u8 depth;
> + u8 csi_dt;
> + u8 check_variants;
> +};
> +
> +static const struct unicam_fmt formats[] = {
> + /* YUV Formats */
> + {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .code = MEDIA_BUS_FMT_YUYV8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .code = MEDIA_BUS_FMT_UYVY8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVYU,
> + .code = MEDIA_BUS_FMT_YVYU8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_VYUY,
> + .code = MEDIA_BUS_FMT_VYUY8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .code = MEDIA_BUS_FMT_YUYV8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVYU,
> + .code = MEDIA_BUS_FMT_YVYU8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_VYUY,
> + .code = MEDIA_BUS_FMT_VYUY8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + /* RGB Formats */
> + .fourcc = V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
> + .depth = 16,
> + .csi_dt = 0x22,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB565X, /* rrrrrggg gggbbbbb */
> + .code = MEDIA_BUS_FMT_RGB565_2X8_BE,
> + .depth = 16,
> + .csi_dt = 0x22
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB555, /* gggbbbbb arrrrrgg */
> + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
> + .depth = 16,
> + .csi_dt = 0x21,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB555X, /* arrrrrgg gggbbbbb */
> + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
> + .depth = 16,
> + .csi_dt = 0x21,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB24, /* rgb */
> + .code = MEDIA_BUS_FMT_RGB888_1X24,
> + .depth = 24,
> + .csi_dt = 0x24,
> + }, {
> + .fourcc = V4L2_PIX_FMT_BGR24, /* bgr */
> + .code = MEDIA_BUS_FMT_BGR888_1X24,
> + .depth = 24,
> + .csi_dt = 0x24,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB32, /* argb */
> + .code = MEDIA_BUS_FMT_ARGB8888_1X32,
> + .depth = 32,
> + .csi_dt = 0x0,
> + }, {
> + /* Bayer Formats */
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + .code = MEDIA_BUS_FMT_SBGGR8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + .code = MEDIA_BUS_FMT_SGBRG8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + .code = MEDIA_BUS_FMT_SGRBG8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + .code = MEDIA_BUS_FMT_SRGGB8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> + .code = MEDIA_BUS_FMT_SBGGR12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> + .code = MEDIA_BUS_FMT_SGBRG12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> + .code = MEDIA_BUS_FMT_SGRBG12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> + .code = MEDIA_BUS_FMT_SRGGB12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR14P,
> + .code = MEDIA_BUS_FMT_SBGGR14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG14P,
> + .code = MEDIA_BUS_FMT_SGBRG14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG14P,
> + .code = MEDIA_BUS_FMT_SGRBG14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB14P,
> + .code = MEDIA_BUS_FMT_SRGGB14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + /*
> + * 16 bit Bayer formats could be supported, but there is no CSI2
> + * data_type defined for raw 16, and no sensors that produce it at
> + * present.
> + */
> +
> + /* Greyscale formats */
> + .fourcc = V4L2_PIX_FMT_GREY,
> + .code = MEDIA_BUS_FMT_Y8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_Y10P,
> + .repacked_fourcc = V4L2_PIX_FMT_Y10,
> + .code = MEDIA_BUS_FMT_Y10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + /* NB There is no packed V4L2 fourcc for this format. */
> + .repacked_fourcc = V4L2_PIX_FMT_Y12,
> + .code = MEDIA_BUS_FMT_Y12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + },
> + /* Embedded data format */
> + {
> + .fourcc = V4L2_META_FMT_SENSOR_DATA,
> + .code = MEDIA_BUS_FMT_SENSOR_DATA,
> + .depth = 8,
> + }
> +};
> +
> +struct unicam_buffer {
> + struct vb2_v4l2_buffer vb;
> + struct list_head list;
> +};
> +
> +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> +{
> + return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> +}
> +
> +struct unicam_node {
> + bool registered;
> + int open;
> + bool streaming;
> + unsigned int pad_id;
> + /* Pointer pointing to current v4l2_buffer */
> + struct unicam_buffer *cur_frm;
> + /* Pointer pointing to next v4l2_buffer */
> + struct unicam_buffer *next_frm;
> + /* video capture */
> + const struct unicam_fmt *fmt;
> + /* Used to store current pixel format */
> + struct v4l2_format v_fmt;
> + /* Used to store current mbus frame format */
> + struct v4l2_mbus_framefmt m_fmt;
> + /* Buffer queue used in video-buf */
> + struct vb2_queue buffer_queue;
> + /* Queue of filled frames */
> + struct list_head dma_queue;
> + /* IRQ lock for DMA queue */
> + spinlock_t dma_queue_lock;
> + /* lock used to access this structure */
> + struct mutex lock;
> + /* Identifies video device for this channel */
> + struct video_device video_dev;
> + /* Pointer to the parent handle */
> + struct unicam_device *dev;
> + struct media_pad pad;
> + unsigned int embedded_lines;
> + /*
> + * Dummy buffer intended to be used by unicam
> + * if we have no other queued buffers to swap to.
> + */
> + void *dummy_buf_cpu_addr;
> + dma_addr_t dummy_buf_dma_addr;
> +};
> +
> +struct unicam_device {
> + struct kref kref;
> +
> + /* V4l2 specific parameters */
> + struct v4l2_async_subdev asd;
> +
> + /* peripheral base address */
> + void __iomem *base;
> + /* clock gating base address */
> + void __iomem *clk_gate_base;
> + /* clock handle */
> + struct clk *clock;
> + /* V4l2 device */
> + struct v4l2_device v4l2_dev;
> + struct media_device mdev;
> +
> + /* parent device */
> + struct platform_device *pdev;
> + /* subdevice async Notifier */
> + struct v4l2_async_notifier notifier;
> + unsigned int sequence;
> +
> + /* ptr to sub device */
> + struct v4l2_subdev *sensor;
> + /* Pad config for the sensor */
> + struct v4l2_subdev_pad_config *sensor_config;
> +
> + enum v4l2_mbus_type bus_type;
> + /*
> + * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> + * bus.mipi_csi1.strobe for CCP2.
> + */
> + unsigned int bus_flags;
> + unsigned int max_data_lanes;
> + unsigned int active_data_lanes;
> + bool sensor_embedded_data;
> +
> + struct unicam_node node[MAX_NODES];
> + struct v4l2_ctrl_handler ctrl_handler;
> +};
> +
> +static inline struct unicam_device *
> +to_unicam_device(struct v4l2_device *v4l2_dev)
> +{
> + return container_of(v4l2_dev, struct unicam_device, v4l2_dev);
> +}
> +
> +/* Hardware access */
> +static inline void clk_write(struct unicam_device *dev, u32 val)
> +{
> + writel(val | 0x5a000000, dev->clk_gate_base);
> +}
> +
> +static inline u32 reg_read(struct unicam_device *dev, u32 offset)
> +{
> + return readl(dev->base + offset);
> +}
> +
> +static inline void reg_write(struct unicam_device *dev, u32 offset, u32 val)
> +{
> + writel(val, dev->base + offset);
> +}
> +
> +static inline int get_field(u32 value, u32 mask)
> +{
> + return (value & mask) >> __ffs(mask);
> +}
> +
> +static inline void set_field(u32 *valp, u32 field, u32 mask)
> +{
> + u32 val = *valp;
> +
> + val &= ~mask;
> + val |= (field << __ffs(mask)) & mask;
> + *valp = val;
> +}
> +
> +static inline u32 reg_read_field(struct unicam_device *dev, u32 offset,
> + u32 mask)
> +{
> + return get_field(reg_read(dev, offset), mask);
> +}
> +
> +static inline void reg_write_field(struct unicam_device *dev, u32 offset,
> + u32 field, u32 mask)
> +{
> + u32 val = reg_read(dev, offset);
> +
> + set_field(&val, field, mask);
> + reg_write(dev, offset, val);
> +}
> +
> +/* Power management functions */
> +static inline int unicam_runtime_get(struct unicam_device *dev)
> +{
> + return pm_runtime_get_sync(&dev->pdev->dev);
> +}
> +
> +static inline void unicam_runtime_put(struct unicam_device *dev)
> +{
> + pm_runtime_put_sync(&dev->pdev->dev);
> +}
> +
> +/* Format setup functions */
> +static const struct unicam_fmt *find_format_by_code(u32 code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> + if (formats[i].code == code)
> + return &formats[i];
> + }
> +
> + return NULL;
> +}
> +
> +static int check_mbus_format(struct unicam_device *dev,
> + const struct unicam_fmt *format)
> +{
> + unsigned int i;
> + int ret = 0;
> +
> + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = i,
> + .pad = IMAGE_PAD,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
> + NULL, &mbus_code);
> +
> + if (!ret && mbus_code.code == format->code)
> + return 1;
> + }
> +
> + return 0;
> +}
> +
> +static const struct unicam_fmt *find_format_by_pix(struct unicam_device *dev,
> + u32 pixelformat)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> + if (formats[i].fourcc == pixelformat ||
> + formats[i].repacked_fourcc == pixelformat) {
> + if (formats[i].check_variants &&
> + !check_mbus_format(dev, &formats[i]))
> + continue;
> + return &formats[i];
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static inline unsigned int bytes_per_line(u32 width,
I'd drop inline, there is no compelling reason for having this as
an inline function IMHO.
> + const struct unicam_fmt *fmt,
> + u32 v4l2_fourcc)
> +{
> + if (v4l2_fourcc == fmt->repacked_fourcc)
> + /* Repacking always goes to 16bpp */
> + return ALIGN(width << 1, BPL_ALIGNMENT);
> + else
> + return ALIGN((width * fmt->depth) >> 3, BPL_ALIGNMENT);
> +}
> +
> +static int __subdev_get_format(struct unicam_device *dev,
> + struct v4l2_mbus_framefmt *fmt, int pad_id)
> +{
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = pad_id
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, get_fmt, dev->sensor_config,
> + &sd_fmt);
> + if (ret < 0)
> + return ret;
> +
> + *fmt = sd_fmt.format;
> +
> + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__,
> + fmt->width, fmt->height, fmt->code);
> +
> + return 0;
> +}
> +
> +static int __subdev_set_format(struct unicam_device *dev,
> + struct v4l2_mbus_framefmt *fmt, int pad_id)
> +{
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = pad_id
> + };
> + int ret;
> +
> + sd_fmt.format = *fmt;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev->sensor_config,
> + &sd_fmt);
> + if (ret < 0)
> + return ret;
> +
> + *fmt = sd_fmt.format;
> +
> + if (pad_id == IMAGE_PAD)
> + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__, fmt->width,
> + fmt->height, fmt->code);
> + else
> + unicam_dbg(1, dev, "%s Embedded data code:%04x\n", __func__,
> + sd_fmt.format.code);
> +
> + return 0;
> +}
> +
> +static int unicam_calc_format_size_bpl(struct unicam_device *dev,
> + const struct unicam_fmt *fmt,
> + struct v4l2_format *f)
> +{
> + unsigned int min_bytesperline;
> +
> + v4l_bound_align_image(&f->fmt.pix.width, MIN_WIDTH, MAX_WIDTH, 2,
> + &f->fmt.pix.height, MIN_HEIGHT, MAX_HEIGHT, 0,
> + 0);
> +
> + min_bytesperline = bytes_per_line(f->fmt.pix.width, fmt,
> + f->fmt.pix.pixelformat);
> +
> + if (f->fmt.pix.bytesperline > min_bytesperline &&
> + f->fmt.pix.bytesperline <= MAX_BYTESPERLINE)
> + f->fmt.pix.bytesperline = ALIGN(f->fmt.pix.bytesperline,
> + BPL_ALIGNMENT);
> + else
> + f->fmt.pix.bytesperline = min_bytesperline;
> +
> + f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
> +
> + unicam_dbg(3, dev, "%s: fourcc: %08X size: %dx%d bpl:%d img_size:%d\n",
> + __func__,
> + f->fmt.pix.pixelformat,
> + f->fmt.pix.width, f->fmt.pix.height,
> + f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
> +
> + return 0;
> +}
> +
> +static int unicam_reset_format(struct unicam_node *node)
> +{
> + struct unicam_device *dev = node->dev;
> + struct v4l2_mbus_framefmt mbus_fmt;
> + int ret;
> +
> + if (dev->sensor_embedded_data || node->pad_id != METADATA_PAD) {
> + ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
> + if (ret) {
> + unicam_err(dev, "Failed to get_format - ret %d\n", ret);
> + return ret;
> + }
> +
> + if (mbus_fmt.code != node->fmt->code) {
> + unicam_err(dev, "code mismatch - fmt->code %08x, mbus_fmt.code %08x\n",
> + node->fmt->code, mbus_fmt.code);
> + return ret;
> + }
> + }
> +
> + if (node->pad_id == IMAGE_PAD) {
> + v4l2_fill_pix_format(&node->v_fmt.fmt.pix, &mbus_fmt);
> + node->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + unicam_calc_format_size_bpl(dev, node->fmt, &node->v_fmt);
> + } else {
> + node->v_fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> + node->v_fmt.fmt.meta.dataformat = V4L2_META_FMT_SENSOR_DATA;
> + if (dev->sensor_embedded_data) {
> + node->v_fmt.fmt.meta.buffersize =
> + mbus_fmt.width * mbus_fmt.height;
> + node->embedded_lines = mbus_fmt.height;
> + } else {
> + node->v_fmt.fmt.meta.buffersize = UNICAM_EMBEDDED_SIZE;
> + node->embedded_lines = 1;
> + }
> + }
> +
> + node->m_fmt = mbus_fmt;
> + return 0;
> +}
> +
> +static void unicam_wr_dma_addr(struct unicam_device *dev, dma_addr_t dmaaddr,
> + unsigned int buffer_size, int pad_id)
> +{
> + dma_addr_t endaddr = dmaaddr + buffer_size;
> +
> + /*
> + * dmaaddr and endaddr should be a 32-bit address with the top two bits
> + * set to 0x3 to signify uncached access through the Videocore memory
> + * controller.
> + */
> + WARN_ON((dmaaddr >> 30) != 0x3 || (endaddr >> 30) != 0x3);
> +
> + if (pad_id == IMAGE_PAD) {
> + reg_write(dev, UNICAM_IBSA0, dmaaddr);
> + reg_write(dev, UNICAM_IBEA0, endaddr);
> + } else {
> + reg_write(dev, UNICAM_DBSA0, dmaaddr);
> + reg_write(dev, UNICAM_DBEA0, endaddr);
> + }
> +}
> +
> +static inline unsigned int unicam_get_lines_done(struct unicam_device *dev)
Same here...
> +{
> + dma_addr_t start_addr, cur_addr;
> + unsigned int stride = dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline;
> + struct unicam_buffer *frm = dev->node[IMAGE_PAD].cur_frm;
> +
> + if (!frm)
> + return 0;
> +
> + start_addr = vb2_dma_contig_plane_dma_addr(&frm->vb.vb2_buf, 0);
> + cur_addr = reg_read(dev, UNICAM_IBWP);
> + return (unsigned int)(cur_addr - start_addr) / stride;
> +}
> +
> +static inline void unicam_schedule_next_buffer(struct unicam_node *node)
...and here.
It's probably a good idea to scan this source for other inlines that do not make sense.
> +{
> + struct unicam_device *dev = node->dev;
> + struct unicam_buffer *buf;
> + unsigned int size;
> + dma_addr_t addr;
> +
> + buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> + node->next_frm = buf;
> + list_del(&buf->list);
> +
> + addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> + size = (node->pad_id == IMAGE_PAD) ?
> + node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> +
> + unicam_wr_dma_addr(dev, addr, size, node->pad_id);
> +}
> +
> +static inline void unicam_schedule_dummy_buffer(struct unicam_node *node)
> +{
> + struct unicam_device *dev = node->dev;
> +
> + unicam_dbg(3, dev, "Scheduling dummy buffer for node %d\n",
> + node->pad_id);
> +
> + unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr, DUMMY_BUF_SIZE,
> + node->pad_id);
> + node->next_frm = NULL;
> +}
> +
> +static inline void unicam_process_buffer_complete(struct unicam_node *node,
> + unsigned int sequence)
> +{
> + node->cur_frm->vb.field = node->m_fmt.field;
> + node->cur_frm->vb.sequence = sequence;
> +
> + vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +static bool unicam_all_nodes_streaming(struct unicam_device *dev)
> +{
> + bool ret;
> +
> + ret = dev->node[IMAGE_PAD].open && dev->node[IMAGE_PAD].streaming;
> + ret &= !dev->node[METADATA_PAD].open ||
> + dev->node[METADATA_PAD].streaming;
> + return ret;
It took me a while to parse this boolean expression, and I'm not sure I like
the mixing of boolean with bitmasks (and in fact, I'm not 100% certain that
that isn't compiler dependent!).
I'd rework this a bit and add a comment.
But I also noticed that this function is used in only one place, so I would
move this code there instead of having this as a separate function.
> +}
> +
> +static bool unicam_all_nodes_disabled(struct unicam_device *dev)
> +{
> + return !dev->node[IMAGE_PAD].streaming &&
> + !dev->node[METADATA_PAD].streaming;
> +}
> +
> +static void unicam_queue_event_sof(struct unicam_device *unicam)
> +{
> + struct v4l2_event event = {
> + .type = V4L2_EVENT_FRAME_SYNC,
> + .u.frame_sync.frame_sequence = unicam->sequence,
> + };
> +
> + v4l2_event_queue(&unicam->node[IMAGE_PAD].video_dev, &event);
> +}
> +
> +/*
> + * unicam_isr : ISR handler for unicam capture
> + * @irq: irq number
> + * @dev_id: dev_id ptr
> + *
> + * It changes status of the captured buffer, takes next buffer from the queue
> + * and sets its address in unicam registers
> + */
> +static irqreturn_t unicam_isr(int irq, void *dev)
> +{
> + struct unicam_device *unicam = dev;
> + unsigned int lines_done = unicam_get_lines_done(dev);
> + unsigned int sequence = unicam->sequence;
> + unsigned int i;
> + u32 ista, sta;
> + u64 ts;
> +
> + /*
> + * Don't service interrupts if not streaming.
> + * Avoids issues if the VPU should enable the
> + * peripheral without the kernel knowing (that
> + * shouldn't happen, but causes issues if it does).
> + */
> + if (unicam_all_nodes_disabled(unicam))
> + return IRQ_NONE;
> +
> + sta = reg_read(unicam, UNICAM_STA);
> + /* Write value back to clear the interrupts */
> + reg_write(unicam, UNICAM_STA, sta);
> +
> + ista = reg_read(unicam, UNICAM_ISTA);
> + /* Write value back to clear the interrupts */
> + reg_write(unicam, UNICAM_ISTA, ista);
> +
> + unicam_dbg(3, unicam, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d",
> + ista, sta, sequence, lines_done);
> +
> + if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> + return IRQ_HANDLED;
> +
> + /*
> + * We must run the frame end handler first. If we have a valid next_frm
> + * and we get a simultaneout FE + FS interrupt, running the FS handler
> + * first would null out the next_frm ptr and we would have lost the
> + * buffer forever.
> + */
> + if (ista & UNICAM_FEI || sta & UNICAM_PI0) {
> + /*
> + * Ensure we have swapped buffers already as we can't
> + * stop the peripheral. If no buffer is available, use a
> + * dummy buffer to dump out frames until we get a new buffer
> + * to use.
> + */
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + if (unicam->node[i].cur_frm)
> + unicam_process_buffer_complete(&unicam->node[i],
> + sequence);
> + unicam->node[i].cur_frm = unicam->node[i].next_frm;
> + }
> + unicam->sequence++;
> + }
> +
> + if (ista & UNICAM_FSI) {
> + /*
> + * Timestamp is to be when the first data byte was captured,
> + * aka frame start.
> + */
> + ts = ktime_get_ns();
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + if (unicam->node[i].cur_frm)
> + unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> + ts;
> + /*
> + * Set the next frame output to go to a dummy frame
> + * if we have not managed to obtain another frame
> + * from the queue.
> + */
> + unicam_schedule_dummy_buffer(&unicam->node[i]);
> + }
> +
> + unicam_queue_event_sof(unicam);
> + }
> +
> + /*
> + * Cannot swap buffer at frame end, there may be a race condition
> + * where the HW does not actually swap it if the new frame has
> + * already started.
> + */
> + if (ista & (UNICAM_FSI | UNICAM_LCI) && !(ista & UNICAM_FEI)) {
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + spin_lock(&unicam->node[i].dma_queue_lock);
> + if (!list_empty(&unicam->node[i].dma_queue) &&
> + !unicam->node[i].next_frm)
> + unicam_schedule_next_buffer(&unicam->node[i]);
> + spin_unlock(&unicam->node[i].dma_queue_lock);
> + }
> + }
> +
> + if (reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> + /* Switch out of trigger mode if selected */
> + reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> + reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> + }
> + return IRQ_HANDLED;
> +}
> +
> +static int unicam_querycap(struct file *file, void *priv,
> + struct v4l2_capability *cap)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + strlcpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> + strlcpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
Use strscpy. The media subsystem has settled on the use of strscpy in
favor of strcpy/strlcpy/strncpy.
> +
> + snprintf(cap->bus_info, sizeof(cap->bus_info),
> + "platform:%s", dev_name(&dev->pdev->dev));
> +
> + cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_fmtdesc *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + unsigned int index = 0;
> + unsigned int i;
> + int ret = 0;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = i,
> + .pad = IMAGE_PAD,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> + const struct unicam_fmt *fmt;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
> + NULL, &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx %d returned %d - index invalid\n",
> + i, ret);
> + return -EINVAL;
> + }
> +
> + fmt = find_format_by_code(mbus_code.code);
> + if (fmt) {
> + if (fmt->fourcc) {
> + if (index == f->index) {
> + f->pixelformat = fmt->fourcc;
> + break;
> + }
> + index++;
> + }
> + if (fmt->repacked_fourcc) {
> + if (index == f->index) {
> + f->pixelformat = fmt->repacked_fourcc;
> + break;
> + }
> + index++;
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int unicam_g_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt = NULL;
> + int ret;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + /*
> + * If a flip has occurred in the sensor, the fmt code might have
> + * changed. So we will need to re-fetch the format from the subdevice.
> + */
> + ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
> + if (ret)
> + return -EINVAL;
> +
> + /* Find the V4L2 format from mbus code. We must match a known format. */
> + fmt = find_format_by_code(mbus_fmt.code);
> + if (!fmt)
> + return -EINVAL;
> +
> + node->fmt = fmt;
> + node->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static
> +const struct unicam_fmt *get_first_supported_format(struct unicam_device *dev)
Strange line break. If you need to split this in two lines, then split it
after the type and function name:
static const struct unicam_fmt *
get_first_supported_format(struct unicam_device *dev)
> +{
> + struct v4l2_subdev_mbus_code_enum mbus_code;
> + const struct unicam_fmt *fmt = NULL;
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; ret != -EINVAL && ret != -ENOIOCTLCMD; ++i) {
> + memset(&mbus_code, 0, sizeof(mbus_code));
> + mbus_code.index = i;
> + mbus_code.pad = IMAGE_PAD;
> + mbus_code.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code, NULL,
> + &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx %u returned %d - continue\n",
> + i, ret);
> + continue;
> + }
> +
> + unicam_dbg(2, dev, "subdev %s: code: 0x%08x idx: %u\n",
> + dev->sensor->name, mbus_code.code, i);
> +
> + fmt = find_format_by_code(mbus_code.code);
> + unicam_dbg(2, dev, "fmt 0x%08x returned as %p, V4L2 FOURCC 0x%08x, csi_dt 0x%02x\n",
> + mbus_code.code, fmt, fmt ? fmt->fourcc : 0,
> + fmt ? fmt->csi_dt : 0);
> + if (fmt)
> + return fmt;
> + }
> +
> + return NULL;
> +}
> +
> +static int unicam_try_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_TRY,
> + .pad = IMAGE_PAD
> + };
> + struct v4l2_mbus_framefmt *mbus_fmt = &sd_fmt.format;
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
> + if (!fmt) {
> + /*
> + * Pixel format not supported by unicam. Choose the first
> + * supported format, and let the sensor choose something else.
> + */
> + unicam_dbg(3, dev, "Fourcc format (0x%08x) not found. Use first format.\n",
> + f->fmt.pix.pixelformat);
> +
> + fmt = &formats[0];
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + }
> +
> + v4l2_fill_mbus_format(mbus_fmt, &f->fmt.pix, fmt->code);
> + /*
> + * No support for receiving interlaced video, so never
> + * request it from the sensor subdev.
> + */
> + mbus_fmt->field = V4L2_FIELD_NONE;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev->sensor_config,
> + &sd_fmt);
> + if (ret && ret != -ENOIOCTLCMD && ret != -ENODEV)
> + return ret;
> +
> + if (mbus_fmt->field != V4L2_FIELD_NONE)
> + unicam_info(dev, "Sensor trying to send interlaced video - results may be unpredictable\n");
> +
> + v4l2_fill_pix_format(&f->fmt.pix, &sd_fmt.format);
> + if (mbus_fmt->code != fmt->code) {
> + /* Sensor has returned an alternate format */
> + fmt = find_format_by_code(mbus_fmt->code);
> + if (!fmt) {
> + /*
> + * The alternate format is one unicam can't support.
> + * Find the first format that is supported by both, and
> + * then set that.
> + */
> + fmt = get_first_supported_format(dev);
> + mbus_fmt->code = fmt->code;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt,
> + dev->sensor_config, &sd_fmt);
> + if (ret && ret != -ENOIOCTLCMD && ret != -ENODEV)
> + return ret;
> +
> + if (mbus_fmt->field != V4L2_FIELD_NONE)
> + unicam_info(dev, "Sensor trying to send interlaced video - results may be unpredictable\n");
> +
> + v4l2_fill_pix_format(&f->fmt.pix, &sd_fmt.format);
> +
> + if (mbus_fmt->code != fmt->code) {
> + /*
> + * We've set a format that the sensor reports
> + * as being supported, but it refuses to set it.
> + * Not much else we can do.
> + * Assume that the sensor driver may accept the
> + * format when it is set (rather than tried).
> + */
> + unicam_err(dev, "Sensor won't accept default format, and Unicam can't support sensor default\n");
> + }
> + }
> +
> + if (fmt->fourcc)
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + else
> + f->fmt.pix.pixelformat = fmt->repacked_fourcc;
> + }
> +
> + return unicam_calc_format_size_bpl(dev, fmt, f);
> +}
> +
> +static int unicam_s_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct vb2_queue *q = &node->buffer_queue;
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (vb2_is_busy(q))
> + return -EBUSY;
> +
> + ret = unicam_try_fmt_vid_cap(file, priv, f);
> + if (ret < 0)
> + return ret;
> +
> + fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
> + if (!fmt) {
> + /*
> + * Unknown pixel format - adopt a default.
> + * This shouldn't happen as try_fmt should have resolved any
> + * issues first.
> + */
> + fmt = get_first_supported_format(dev);
> + if (!fmt)
> + /*
> + * It shouldn't be possible to get here with no
> + * supported formats
> + */
> + return -EINVAL;
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + return -EINVAL;
> + }
> +
> + v4l2_fill_mbus_format(&mbus_fmt, &f->fmt.pix, fmt->code);
> +
> + ret = __subdev_set_format(dev, &mbus_fmt, node->pad_id);
> + if (ret) {
> + unicam_dbg(3, dev, "%s __subdev_set_format failed %d\n",
> + __func__, ret);
> + return ret;
> + }
> +
> + /* Just double check nothing has gone wrong */
> + if (mbus_fmt.code != fmt->code) {
> + unicam_dbg(3, dev,
> + "%s subdev changed format on us, this should not happen\n",
> + __func__);
> + return -EINVAL;
> + }
> +
> + node->fmt = fmt;
> + node->v_fmt.fmt.pix.pixelformat = f->fmt.pix.pixelformat;
> + node->v_fmt.fmt.pix.bytesperline = f->fmt.pix.bytesperline;
> + unicam_reset_format(node);
> +
> + unicam_dbg(3, dev,
> + "%s %dx%d, mbus_fmt 0x%08X, V4L2 pix 0x%08X.\n",
> + __func__, node->v_fmt.fmt.pix.width,
> + node->v_fmt.fmt.pix.height, mbus_fmt.code,
> + node->v_fmt.fmt.pix.pixelformat);
> +
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_fmt_meta_cap(struct file *file, void *priv,
> + struct v4l2_fmtdesc *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + u32 code;
> + int ret = 0;
> +
> + if (node->pad_id != METADATA_PAD || f->index != 0)
> + return -EINVAL;
> +
> + if (dev->sensor_embedded_data) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = f->index,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = METADATA_PAD,
> + };
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code, NULL,
> + &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx 0 returned %d - index invalid\n",
> + ret);
> + return -EINVAL;
> + }
> +
> + code = mbus_code.code;
> + } else {
> + code = MEDIA_BUS_FMT_SENSOR_DATA;
> + }
> +
> + fmt = find_format_by_code(code);
> + if (fmt)
> + f->pixelformat = fmt->fourcc;
> +
> + return 0;
> +}
> +
> +static int unicam_g_fmt_meta_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> +
> + if (node->pad_id != METADATA_PAD)
> + return -EINVAL;
> +
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static int unicam_queue_setup(struct vb2_queue *vq,
> + unsigned int *nbuffers,
> + unsigned int *nplanes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> + unsigned int size = node->pad_id == IMAGE_PAD ?
> + node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> +
> + if (vq->num_buffers + *nbuffers < 3)
> + *nbuffers = 3 - vq->num_buffers;
> +
> + if (*nplanes) {
> + if (sizes[0] < size) {
> + unicam_err(dev, "sizes[0] %i < size %u\n", sizes[0],
> + size);
> + return -EINVAL;
> + }
> + size = sizes[0];
> + }
> +
> + *nplanes = 1;
> + sizes[0] = size;
> +
> + return 0;
> +}
> +
> +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct unicam_device *dev = node->dev;
> + struct unicam_buffer *buf = to_unicam_buffer(vb);
> + unsigned long size;
> +
> + if (WARN_ON(!node->fmt))
> + return -EINVAL;
> +
> + size = node->pad_id == IMAGE_PAD ? node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> + if (vb2_plane_size(vb, 0) < size) {
> + unicam_err(dev, "data will not fit into plane (%lu < %lu)\n",
> + vb2_plane_size(vb, 0), size);
> + return -EINVAL;
> + }
> +
> + vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> + return 0;
> +}
> +
> +static void unicam_buffer_queue(struct vb2_buffer *vb)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct unicam_buffer *buf = to_unicam_buffer(vb);
> + unsigned long flags;
> +
> + spin_lock_irqsave(&node->dma_queue_lock, flags);
> + list_add_tail(&buf->list, &node->dma_queue);
> + spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +}
> +
> +static void unicam_set_packing_config(struct unicam_device *dev)
> +{
> + u32 pack, unpack;
> + u32 val;
> +
> + if (dev->node[IMAGE_PAD].v_fmt.fmt.pix.pixelformat ==
> + dev->node[IMAGE_PAD].fmt->fourcc) {
> + unpack = UNICAM_PUM_NONE;
> + pack = UNICAM_PPM_NONE;
> + } else {
> + switch (dev->node[IMAGE_PAD].fmt->depth) {
> + case 8:
> + unpack = UNICAM_PUM_UNPACK8;
> + break;
> + case 10:
> + unpack = UNICAM_PUM_UNPACK10;
> + break;
> + case 12:
> + unpack = UNICAM_PUM_UNPACK12;
> + break;
> + case 14:
> + unpack = UNICAM_PUM_UNPACK14;
> + break;
> + case 16:
> + unpack = UNICAM_PUM_UNPACK16;
> + break;
> + default:
> + unpack = UNICAM_PUM_NONE;
> + break;
> + }
> +
> + /* Repacking is always to 16bpp */
> + pack = UNICAM_PPM_PACK16;
> + }
> +
> + val = 0;
> + set_field(&val, unpack, UNICAM_PUM_MASK);
> + set_field(&val, pack, UNICAM_PPM_MASK);
> + reg_write(dev, UNICAM_IPIPE, val);
> +}
> +
> +static void unicam_cfg_image_id(struct unicam_device *dev)
> +{
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 mode, hardcode VC 0 for now. */
> + reg_write(dev, UNICAM_IDI0,
> + (0 << 6) | dev->node[IMAGE_PAD].fmt->csi_dt);
> + } else {
> + /* CCP2 mode */
> + reg_write(dev, UNICAM_IDI0,
> + 0x80 | dev->node[IMAGE_PAD].fmt->csi_dt);
> + }
> +}
> +
> +static void unicam_enable_ed(struct unicam_device *dev)
> +{
> + u32 val = reg_read(dev, UNICAM_DCS);
> +
> + set_field(&val, 2, UNICAM_EDL_MASK);
> + /* Do not wrap at the end of the embedded data buffer */
> + set_field(&val, 0, UNICAM_DBOB);
> +
> + reg_write(dev, UNICAM_DCS, val);
> +}
> +
> +static void unicam_start_rx(struct unicam_device *dev, dma_addr_t *addr)
> +{
> + int line_int_freq = dev->node[IMAGE_PAD].v_fmt.fmt.pix.height >> 2;
> + unsigned int size, i;
> + u32 val;
> +
> + if (line_int_freq < 128)
> + line_int_freq = 128;
> +
> + /* Enable lane clocks */
> + val = 1;
> + for (i = 0; i < dev->active_data_lanes; i++)
> + val = val << 2 | 1;
> + clk_write(dev, val);
> +
> + /* Basic init */
> + reg_write(dev, UNICAM_CTRL, UNICAM_MEM);
> +
> + /* Enable analogue control, and leave in reset. */
> + val = UNICAM_AR;
> + set_field(&val, 7, UNICAM_CTATADJ_MASK);
> + set_field(&val, 7, UNICAM_PTATADJ_MASK);
> + reg_write(dev, UNICAM_ANA, val);
> + usleep_range(1000, 2000);
> +
> + /* Come out of reset */
> + reg_write_field(dev, UNICAM_ANA, 0, UNICAM_AR);
> +
> + /* Peripheral reset */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> + /* Enable Rx control. */
> + val = reg_read(dev, UNICAM_CTRL);
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> + set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> + } else {
> + set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> + set_field(&val, dev->bus_flags, UNICAM_DCM_MASK);
> + }
> + /* Packet framer timeout */
> + set_field(&val, 0xf, UNICAM_PFT_MASK);
> + set_field(&val, 128, UNICAM_OET_MASK);
> + reg_write(dev, UNICAM_CTRL, val);
> +
> + reg_write(dev, UNICAM_IHWIN, 0);
> + reg_write(dev, UNICAM_IVWIN, 0);
> +
> + /* AXI bus access QoS setup */
> + val = reg_read(dev, UNICAM_PRI);
> + set_field(&val, 0, UNICAM_BL_MASK);
> + set_field(&val, 0, UNICAM_BS_MASK);
> + set_field(&val, 0xe, UNICAM_PP_MASK);
> + set_field(&val, 8, UNICAM_NP_MASK);
> + set_field(&val, 2, UNICAM_PT_MASK);
> + set_field(&val, 1, UNICAM_PE);
> + reg_write(dev, UNICAM_PRI, val);
> +
> + reg_write_field(dev, UNICAM_ANA, 0, UNICAM_DDL);
> +
> + /* Always start in trigger frame capture mode (UNICAM_FCM set) */
> + val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> + set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> + reg_write(dev, UNICAM_ICTL, val);
> + reg_write(dev, UNICAM_STA, UNICAM_STA_MASK_ALL);
> + reg_write(dev, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> +
> + /* tclk_term_en */
> + reg_write_field(dev, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> + /* tclk_settle */
> + reg_write_field(dev, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> + /* td_term_en */
> + reg_write_field(dev, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> + /* ths_settle */
> + reg_write_field(dev, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> + /* trx_enable */
> + reg_write_field(dev, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> +
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_SOE);
> +
> + /* Packet compare setup - required to avoid missing frame ends */
> + val = 0;
> + set_field(&val, 1, UNICAM_PCE);
> + set_field(&val, 1, UNICAM_GI);
> + set_field(&val, 1, UNICAM_CPH);
> + set_field(&val, 0, UNICAM_PCVC_MASK);
> + set_field(&val, 1, UNICAM_PCDT_MASK);
> + reg_write(dev, UNICAM_CMP0, val);
> +
> + /* Enable clock lane and set up terminations */
> + val = 0;
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 */
> + set_field(&val, 1, UNICAM_CLE);
> + set_field(&val, 1, UNICAM_CLLPE);
> + if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
> + set_field(&val, 1, UNICAM_CLTRE);
> + set_field(&val, 1, UNICAM_CLHSE);
> + }
> + } else {
> + /* CCP2 */
> + set_field(&val, 1, UNICAM_CLE);
> + set_field(&val, 1, UNICAM_CLHSE);
> + set_field(&val, 1, UNICAM_CLTRE);
> + }
> + reg_write(dev, UNICAM_CLK, val);
> +
> + /*
> + * Enable required data lanes with appropriate terminations.
> + * The same value needs to be written to UNICAM_DATn registers for
> + * the active lanes, and 0 for inactive ones.
> + */
> + val = 0;
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 */
> + set_field(&val, 1, UNICAM_DLE);
> + set_field(&val, 1, UNICAM_DLLPE);
> + if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
> + set_field(&val, 1, UNICAM_DLTRE);
> + set_field(&val, 1, UNICAM_DLHSE);
> + }
> + } else {
> + /* CCP2 */
> + set_field(&val, 1, UNICAM_DLE);
> + set_field(&val, 1, UNICAM_DLHSE);
> + set_field(&val, 1, UNICAM_DLTRE);
> + }
> + reg_write(dev, UNICAM_DAT0, val);
> +
> + if (dev->active_data_lanes == 1)
> + val = 0;
> + reg_write(dev, UNICAM_DAT1, val);
> +
> + if (dev->max_data_lanes > 2) {
> + /*
> + * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> + * instance supports more than 2 data lanes.
> + */
> + if (dev->active_data_lanes == 2)
> + val = 0;
> + reg_write(dev, UNICAM_DAT2, val);
> +
> + if (dev->active_data_lanes == 3)
> + val = 0;
> + reg_write(dev, UNICAM_DAT3, val);
> + }
> +
> + reg_write(dev, UNICAM_IBLS,
> + dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline);
> + size = dev->node[IMAGE_PAD].v_fmt.fmt.pix.sizeimage;
> + unicam_wr_dma_addr(dev, addr[IMAGE_PAD], size, IMAGE_PAD);
> + unicam_set_packing_config(dev);
> + unicam_cfg_image_id(dev);
> +
> + val = reg_read(dev, UNICAM_MISC);
> + set_field(&val, 1, UNICAM_FL0);
> + set_field(&val, 1, UNICAM_FL1);
> + reg_write(dev, UNICAM_MISC, val);
> +
> + if (dev->node[METADATA_PAD].streaming && dev->sensor_embedded_data) {
> + size = dev->node[METADATA_PAD].v_fmt.fmt.meta.buffersize;
> + unicam_enable_ed(dev);
> + unicam_wr_dma_addr(dev, addr[METADATA_PAD], size, METADATA_PAD);
> + }
> +
> + /* Enable peripheral */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPE);
> +
> + /* Load image pointers */
> + reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> +
> + /* Load embedded data buffer pointers if needed */
> + if (dev->node[METADATA_PAD].streaming && dev->sensor_embedded_data)
> + reg_write_field(dev, UNICAM_DCS, 1, UNICAM_LDP);
> +
> + /*
> + * Enable trigger only for the first frame to
> + * sync correctly to the FS from the source.
> + */
> + reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_TFC);
> +}
> +
> +static void unicam_disable(struct unicam_device *dev)
> +{
> + /* Analogue lane control disable */
> + reg_write_field(dev, UNICAM_ANA, 1, UNICAM_DDL);
> +
> + /* Stop the output engine */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_SOE);
> +
> + /* Disable the data lanes. */
> + reg_write(dev, UNICAM_DAT0, 0);
> + reg_write(dev, UNICAM_DAT1, 0);
> +
> + if (dev->max_data_lanes > 2) {
> + reg_write(dev, UNICAM_DAT2, 0);
> + reg_write(dev, UNICAM_DAT3, 0);
> + }
> +
> + /* Peripheral reset */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
> + usleep_range(50, 100);
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> + /* Disable peripheral */
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> + /* Clear ED setup */
> + reg_write(dev, UNICAM_DCS, 0);
> +
> + /* Disable all lane clocks */
> + clk_write(dev, 0);
> +}
> +
> +static void unicam_return_buffers(struct unicam_node *node)
> +{
> + struct unicam_buffer *buf, *tmp;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&node->dma_queue_lock, flags);
> + list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> + list_del(&buf->list);
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> + }
> +
> + if (node->cur_frm)
> + vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> + VB2_BUF_STATE_ERROR);
> + if (node->next_frm && node->cur_frm != node->next_frm)
> + vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> + VB2_BUF_STATE_ERROR);
> +
> + node->cur_frm = NULL;
> + node->next_frm = NULL;
> + spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +}
> +
> +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> + dma_addr_t buffer_addr[MAX_NODES] = { 0 };
> + unsigned long flags;
> + unsigned int i;
> + int ret;
> +
> + node->streaming = true;
> + if (!unicam_all_nodes_streaming(dev)) {
As suggested above, just move the actual code of that function here.
It certainly makes it easier to review.
> + unicam_dbg(3, dev, "Not all nodes are streaming yet.");
> + return 0;
> + }
> +
> + dev->sequence = 0;
> + ret = unicam_runtime_get(dev);
> + if (ret < 0) {
> + unicam_dbg(3, dev, "unicam_runtime_get failed\n");
> + goto err_streaming;
> + }
> +
> + /*
> + * TODO: Retrieve the number of active data lanes from the connected
> + * subdevice.
> + */
> + dev->active_data_lanes = dev->max_data_lanes;
> +
> + ret = clk_set_rate(dev->clock, 100 * 1000 * 1000);
> + if (ret) {
> + unicam_err(dev, "failed to set up clock\n");
> + goto err_pm_put;
> + }
> +
> + ret = clk_prepare_enable(dev->clock);
> + if (ret) {
> + unicam_err(dev, "Failed to enable CSI clock: %d\n", ret);
> + goto err_pm_put;
> + }
> +
> + for (i = 0; i < ARRAY_SIZE(dev->node); i++) {
> + struct unicam_buffer *buf;
> +
> + if (!dev->node[i].streaming)
> + continue;
> +
> + spin_lock_irqsave(&dev->node[i].dma_queue_lock, flags);
> + buf = list_first_entry(&dev->node[i].dma_queue,
> + struct unicam_buffer, list);
> + dev->node[i].cur_frm = buf;
> + dev->node[i].next_frm = buf;
> + list_del(&buf->list);
> + spin_unlock_irqrestore(&dev->node[i].dma_queue_lock, flags);
> +
> + buffer_addr[i] =
> + vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> + }
> +
> + unicam_start_rx(dev, buffer_addr);
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_stream, 1);
> + if (ret < 0) {
> + unicam_err(dev, "stream on failed in subdev\n");
> + goto err_disable_unicam;
> + }
> +
> + return 0;
> +
> +err_disable_unicam:
> + unicam_disable(dev);
> + clk_disable_unprepare(dev->clock);
> +err_pm_put:
> + unicam_runtime_put(dev);
> +err_streaming:
> + unicam_return_buffers(node);
That function returns the buffers with state ERROR, but in start_streaming that
should be state DEQUEUED.
> + node->streaming = false;
> +
> + return ret;
> +}
> +
> +static void unicam_stop_streaming(struct vb2_queue *vq)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> +
> + node->streaming = false;
> +
> + if (node->pad_id == IMAGE_PAD) {
> + /*
> + * Stop streaming the sensor and disable the peripheral.
> + * We cannot continue streaming embedded data with the
> + * image pad disabled.
> + */
> + if (v4l2_subdev_call(dev->sensor, video, s_stream, 0) < 0)
> + unicam_err(dev, "stream off failed in subdev\n");
> +
> + unicam_disable(dev);
> + clk_disable_unprepare(dev->clock);
> + unicam_runtime_put(dev);
> +
> + } else if (node->pad_id == METADATA_PAD) {
> + /*
> + * Allow the hardware to spin in the dummy buffer.
> + * This is only really needed if the embedded data pad is
> + * disabled before the image pad.
> + */
> + unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr,
> + DUMMY_BUF_SIZE, METADATA_PAD);
> + }
> +
> + /* Clear all queued buffers for the node */
> + unicam_return_buffers(node);
> +}
> +
> +static int unicam_enum_input(struct file *file, void *priv,
> + struct v4l2_input *inp)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + if (inp->index != 0)
> + return -EINVAL;
> +
> + inp->type = V4L2_INPUT_TYPE_CAMERA;
> + if (v4l2_subdev_has_op(dev->sensor, video, s_dv_timings)) {
> + inp->capabilities = V4L2_IN_CAP_DV_TIMINGS;
> + inp->std = 0;
> + } else if (v4l2_subdev_has_op(dev->sensor, video, s_std)) {
> + inp->capabilities = V4L2_IN_CAP_STD;
> + if (v4l2_subdev_call(dev->sensor, video, g_tvnorms, &inp->std)
> + < 0)
Just put this in one line since this is hard to read.
> + inp->std = V4L2_STD_ALL;
> + } else {
> + inp->capabilities = 0;
> + inp->std = 0;
> + }
> + sprintf(inp->name, "Camera 0");
snprintf
> + return 0;
> +}
> +
> +static int unicam_g_input(struct file *file, void *priv, unsigned int *i)
> +{
> + *i = 0;
> +
> + return 0;
> +}
> +
> +static int unicam_s_input(struct file *file, void *priv, unsigned int i)
> +{
> + /*
> + * FIXME: Ideally we would like to be able to query the source
> + * subdevice for information over the input connectors it supports,
> + * and map that through in to a call to video_ops->s_routing.
> + * There is no infrastructure support for defining that within
> + * devicetree at present. Until that is implemented we can't
> + * map a user physical connector number to s_routing input number.
> + */
> + if (i > 0)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int unicam_querystd(struct file *file, void *priv,
> + v4l2_std_id *std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, querystd, std);
> +}
> +
> +static int unicam_g_std(struct file *file, void *priv, v4l2_std_id *std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, g_std, std);
> +}
> +
> +static int unicam_s_std(struct file *file, void *priv, v4l2_std_id std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + int ret;
> + v4l2_std_id current_std;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, g_std, ¤t_std);
> + if (ret)
> + return ret;
> +
> + if (std == current_std)
> + return 0;
> +
> + if (vb2_is_busy(&node->buffer_queue))
> + return -EBUSY;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_std, std);
> +
> + /* Force recomputation of bytesperline */
> + node->v_fmt.fmt.pix.bytesperline = 0;
> +
> + unicam_reset_format(node);
> +
> + return ret;
> +}
> +
> +static int unicam_s_edid(struct file *file, void *priv, struct v4l2_edid *edid)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, set_edid, edid);
> +}
> +
> +static int unicam_g_edid(struct file *file, void *priv, struct v4l2_edid *edid)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, get_edid, edid);
> +}
> +
> +static int unicam_s_selection(struct file *file, void *priv,
> + struct v4l2_selection *sel)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_selection sdsel = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .target = sel->target,
> + .flags = sel->flags,
> + .r = sel->r,
> + };
> +
> + return v4l2_subdev_call(dev->sensor, pad, set_selection, NULL, &sdsel);
> +}
> +
> +static int unicam_g_selection(struct file *file, void *priv,
> + struct v4l2_selection *sel)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_selection sdsel = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .target = sel->target,
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, get_selection, NULL, &sdsel);
> + if (!ret)
> + sel->r = sdsel.r;
> +
> + return ret;
> +}
> +
> +static int unicam_enum_framesizes(struct file *file, void *priv,
> + struct v4l2_frmsizeenum *fsize)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + struct v4l2_subdev_frame_size_enum fse;
> + int ret;
> +
> + /* check for valid format */
> + fmt = find_format_by_pix(dev, fsize->pixel_format);
> + if (!fmt) {
> + unicam_dbg(3, dev, "Invalid pixel code: %x\n",
> + fsize->pixel_format);
> + return -EINVAL;
> + }
> + fse.code = fmt->code;
> +
> + fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> + fse.index = fsize->index;
> + fse.pad = node->pad_id;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_size, NULL, &fse);
> + if (ret)
> + return ret;
> +
> + unicam_dbg(1, dev, "%s: index: %d code: %x W:[%d,%d] H:[%d,%d]\n",
> + __func__, fse.index, fse.code, fse.min_width, fse.max_width,
> + fse.min_height, fse.max_height);
> +
> + fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
> + fsize->discrete.width = fse.max_width;
> + fsize->discrete.height = fse.max_height;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_frameintervals(struct file *file, void *priv,
> + struct v4l2_frmivalenum *fival)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + struct v4l2_subdev_frame_interval_enum fie = {
> + .index = fival->index,
> + .width = fival->width,
> + .height = fival->height,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> + int ret;
> +
> + fmt = find_format_by_pix(dev, fival->pixel_format);
> + if (!fmt)
> + return -EINVAL;
> +
> + fie.code = fmt->code;
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_interval,
> + NULL, &fie);
> + if (ret)
> + return ret;
> +
> + fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
> + fival->discrete = fie.interval;
> +
> + return 0;
> +}
> +
> +static int unicam_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_g_parm_cap(video_devdata(file), dev->sensor, a);
> +}
> +
> +static int unicam_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_s_parm_cap(video_devdata(file), dev->sensor, a);
> +}
> +
> +static int unicam_g_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, g_dv_timings, timings);
> +}
> +
> +static int unicam_s_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_dv_timings current_timings;
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, g_dv_timings,
> + ¤t_timings);
ret isn't checked!
> +
> + if (v4l2_match_dv_timings(timings, ¤t_timings, 0, false))
> + return 0;
> +
> + if (vb2_is_busy(&node->buffer_queue))
> + return -EBUSY;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_dv_timings, timings);
> +
> + /* Force recomputation of bytesperline */
> + node->v_fmt.fmt.pix.bytesperline = 0;
> +
> + unicam_reset_format(node);
> +
> + return ret;
> +}
> +
> +static int unicam_query_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, query_dv_timings, timings);
> +}
> +
> +static int unicam_enum_dv_timings(struct file *file, void *priv,
> + struct v4l2_enum_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, enum_dv_timings, timings);
> +}
> +
> +static int unicam_dv_timings_cap(struct file *file, void *priv,
> + struct v4l2_dv_timings_cap *cap)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, dv_timings_cap, cap);
> +}
> +
> +static int unicam_subscribe_event(struct v4l2_fh *fh,
> + const struct v4l2_event_subscription *sub)
> +{
> + switch (sub->type) {
> + case V4L2_EVENT_FRAME_SYNC:
> + return v4l2_event_subscribe(fh, sub, 2, NULL);
> + case V4L2_EVENT_SOURCE_CHANGE:
> + return v4l2_event_subscribe(fh, sub, 4, NULL);
> + }
> +
> + return v4l2_ctrl_subscribe_event(fh, sub);
> +}
> +
> +static int unicam_log_status(struct file *file, void *fh)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + u32 reg;
> +
> + /* status for sub devices */
> + v4l2_device_call_all(&dev->v4l2_dev, 0, core, log_status);
> +
> + unicam_info(dev, "-----Receiver status-----\n");
> + unicam_info(dev, "V4L2 width/height: %ux%u\n",
> + node->v_fmt.fmt.pix.width, node->v_fmt.fmt.pix.height);
> + unicam_info(dev, "Mediabus format: %08x\n", node->fmt->code);
> + unicam_info(dev, "V4L2 format: %08x\n",
> + node->v_fmt.fmt.pix.pixelformat);
> + reg = reg_read(dev, UNICAM_IPIPE);
> + unicam_info(dev, "Unpacking/packing: %u / %u\n",
> + get_field(reg, UNICAM_PUM_MASK),
> + get_field(reg, UNICAM_PPM_MASK));
> + unicam_info(dev, "----Live data----\n");
> + unicam_info(dev, "Programmed stride: %4u\n",
> + reg_read(dev, UNICAM_IBLS));
> + unicam_info(dev, "Detected resolution: %ux%u\n",
> + reg_read(dev, UNICAM_IHSTA),
> + reg_read(dev, UNICAM_IVSTA));
> + unicam_info(dev, "Write pointer: %08x\n",
> + reg_read(dev, UNICAM_IBWP));
> +
> + return 0;
> +}
> +
> +static void unicam_notify(struct v4l2_subdev *sd,
> + unsigned int notification, void *arg)
> +{
> + struct unicam_device *dev = to_unicam_device(sd->v4l2_dev);
> +
> + switch (notification) {
> + case V4L2_DEVICE_NOTIFY_EVENT:
> + v4l2_event_queue(&dev->node[IMAGE_PAD].video_dev, arg);
> + break;
> + default:
> + break;
> + }
> +}
> +
> +static const struct vb2_ops unicam_video_qops = {
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .queue_setup = unicam_queue_setup,
> + .buf_prepare = unicam_buffer_prepare,
> + .buf_queue = unicam_buffer_queue,
> + .start_streaming = unicam_start_streaming,
> + .stop_streaming = unicam_stop_streaming,
> +};
> +
> +/*
> + * unicam_v4l2_open : This function is based on the v4l2_fh_open helper
> + * function. It has been augmented to handle sensor subdevice power management,
> + */
> +static int unicam_v4l2_open(struct file *file)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + int ret;
> +
> + mutex_lock(&node->lock);
> +
> + ret = v4l2_fh_open(file);
> + if (ret) {
> + unicam_err(dev, "v4l2_fh_open failed\n");
> + goto unlock;
> + }
> +
> + node->open++;
> +
> + if (!v4l2_fh_is_singular_file(file))
> + goto unlock;
> +
> + ret = v4l2_subdev_call(dev->sensor, core, s_power, 1);
> + if (ret < 0 && ret != -ENOIOCTLCMD) {
> + v4l2_fh_release(file);
> + node->open--;
> + goto unlock;
> + }
> +
> + ret = 0;
> +
> +unlock:
> + mutex_unlock(&node->lock);
> + return ret;
> +}
> +
> +static int unicam_v4l2_release(struct file *file)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev *sd = dev->sensor;
> + bool fh_singular;
> + int ret;
> +
> + mutex_lock(&node->lock);
> +
> + fh_singular = v4l2_fh_is_singular_file(file);
> +
> + ret = _vb2_fop_release(file, NULL);
> +
> + if (fh_singular)
> + v4l2_subdev_call(sd, core, s_power, 0);
> +
> + node->open--;
> + mutex_unlock(&node->lock);
> +
> + return ret;
> +}
> +
> +/* unicam capture driver file operations */
> +static const struct v4l2_file_operations unicam_fops = {
> + .owner = THIS_MODULE,
> + .open = unicam_v4l2_open,
> + .release = unicam_v4l2_release,
> + .read = vb2_fop_read,
> + .poll = vb2_fop_poll,
> + .unlocked_ioctl = video_ioctl2,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +/* unicam capture ioctl operations */
> +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> + .vidioc_querycap = unicam_querycap,
> + .vidioc_enum_fmt_vid_cap = unicam_enum_fmt_vid_cap,
> + .vidioc_g_fmt_vid_cap = unicam_g_fmt_vid_cap,
> + .vidioc_s_fmt_vid_cap = unicam_s_fmt_vid_cap,
> + .vidioc_try_fmt_vid_cap = unicam_try_fmt_vid_cap,
> +
> + .vidioc_enum_fmt_meta_cap = unicam_enum_fmt_meta_cap,
> + .vidioc_g_fmt_meta_cap = unicam_g_fmt_meta_cap,
> + .vidioc_s_fmt_meta_cap = unicam_g_fmt_meta_cap,
> + .vidioc_try_fmt_meta_cap = unicam_g_fmt_meta_cap,
> +
> + .vidioc_enum_input = unicam_enum_input,
> + .vidioc_g_input = unicam_g_input,
> + .vidioc_s_input = unicam_s_input,
> +
> + .vidioc_querystd = unicam_querystd,
> + .vidioc_s_std = unicam_s_std,
> + .vidioc_g_std = unicam_g_std,
> +
> + .vidioc_g_edid = unicam_g_edid,
> + .vidioc_s_edid = unicam_s_edid,
> +
> + .vidioc_enum_framesizes = unicam_enum_framesizes,
> + .vidioc_enum_frameintervals = unicam_enum_frameintervals,
> +
> + .vidioc_g_selection = unicam_g_selection,
> + .vidioc_s_selection = unicam_s_selection,
> +
> + .vidioc_g_parm = unicam_g_parm,
> + .vidioc_s_parm = unicam_s_parm,
> +
> + .vidioc_s_dv_timings = unicam_s_dv_timings,
> + .vidioc_g_dv_timings = unicam_g_dv_timings,
> + .vidioc_query_dv_timings = unicam_query_dv_timings,
> + .vidioc_enum_dv_timings = unicam_enum_dv_timings,
> + .vidioc_dv_timings_cap = unicam_dv_timings_cap,
> +
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +
> + .vidioc_log_status = unicam_log_status,
> + .vidioc_subscribe_event = unicam_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static int
> +unicam_async_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_subdev *asd)
> +{
> + struct unicam_device *unicam = to_unicam_device(notifier->v4l2_dev);
> +
> + if (unicam->sensor) {
> + unicam_info(unicam, "Rejecting subdev %s (Already set!!)",
> + subdev->name);
> + return 0;
> + }
> +
> + unicam->sensor = subdev;
> + unicam_dbg(1, unicam, "Using sensor %s for capture\n", subdev->name);
> +
> + return 0;
> +}
> +
> +static void unicam_release(struct kref *kref)
> +{
> + struct unicam_device *unicam =
> + container_of(kref, struct unicam_device, kref);
> +
> + v4l2_ctrl_handler_free(&unicam->ctrl_handler);
> + media_device_cleanup(&unicam->mdev);
> +
> + if (unicam->sensor_config)
> + v4l2_subdev_free_pad_config(unicam->sensor_config);
> +
> + kfree(unicam);
> +}
> +
> +static void unicam_put(struct unicam_device *unicam)
> +{
> + kref_put(&unicam->kref, unicam_release);
> +}
> +
> +static void unicam_get(struct unicam_device *unicam)
> +{
> + kref_get(&unicam->kref);
> +}
> +
> +static void unicam_node_release(struct video_device *vdev)
> +{
> + struct unicam_node *node = video_get_drvdata(vdev);
> +
> + unicam_put(node->dev);
> +}
> +
> +static int register_node(struct unicam_device *unicam, struct unicam_node *node,
> + enum v4l2_buf_type type, int pad_id)
> +{
> + struct video_device *vdev;
> + struct vb2_queue *q;
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (pad_id == IMAGE_PAD) {
> + ret = __subdev_get_format(unicam, &mbus_fmt, pad_id);
> + if (ret) {
> + unicam_err(unicam, "Failed to get_format - ret %d\n",
> + ret);
> + return ret;
> + }
> +
> + fmt = find_format_by_code(mbus_fmt.code);
> + if (!fmt) {
> + /*
> + * Find the first format that the sensor and unicam both
> + * support
> + */
> + fmt = get_first_supported_format(unicam);
> +
> + if (!fmt)
> + /* No compatible formats */
> + return -EINVAL;
> +
> + mbus_fmt.code = fmt->code;
> + ret = __subdev_set_format(unicam, &mbus_fmt, pad_id);
> + if (ret)
> + return -EINVAL;
> + }
> + if (mbus_fmt.field != V4L2_FIELD_NONE) {
> + /* Interlaced not supported - disable it now. */
> + mbus_fmt.field = V4L2_FIELD_NONE;
> + ret = __subdev_set_format(unicam, &mbus_fmt, pad_id);
> + if (ret)
> + return -EINVAL;
> + }
> +
> + node->v_fmt.fmt.pix.pixelformat = fmt->fourcc ? fmt->fourcc
> + : fmt->repacked_fourcc;
> + } else {
> + /* Fix this node format as embedded data. */
> + fmt = find_format_by_code(MEDIA_BUS_FMT_SENSOR_DATA);
> + node->v_fmt.fmt.meta.dataformat = fmt->fourcc;
> + }
> +
> + node->dev = unicam;
> + node->pad_id = pad_id;
> + node->fmt = fmt;
> +
> + /* Read current subdev format */
> + unicam_reset_format(node);
> +
> + if (v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
> + v4l2_std_id tvnorms;
> +
> + if (WARN_ON(!v4l2_subdev_has_op(unicam->sensor, video,
> + g_tvnorms)))
> + /*
> + * Subdevice should not advertise s_std but not
> + * g_tvnorms
> + */
> + return -EINVAL;
> +
> + ret = v4l2_subdev_call(unicam->sensor, video,
> + g_tvnorms, &tvnorms);
> + if (WARN_ON(ret))
> + return -EINVAL;
> + node->video_dev.tvnorms |= tvnorms;
> + }
> +
> + spin_lock_init(&node->dma_queue_lock);
> + mutex_init(&node->lock);
> +
> + vdev = &node->video_dev;
> + if (pad_id == IMAGE_PAD) {
> + /* Add controls from the subdevice */
> + ret = v4l2_ctrl_add_handler(&unicam->ctrl_handler,
> + unicam->sensor->ctrl_handler, NULL,
> + true);
> + if (ret < 0)
> + return ret;
> +
> + /*
> + * If the sensor subdevice has any controls, associate the node
> + * with the ctrl handler to allow access from userland.
> + */
> + if (!list_empty(&unicam->ctrl_handler.ctrls))
> + vdev->ctrl_handler = &unicam->ctrl_handler;
> + }
> +
> + q = &node->buffer_queue;
> + q->type = type;
> + q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;
> + q->drv_priv = node;
> + q->ops = &unicam_video_qops;
> + q->mem_ops = &vb2_dma_contig_memops;
> + q->buf_struct_size = sizeof(struct unicam_buffer);
> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + q->lock = &node->lock;
> + q->min_buffers_needed = 2;
> + q->dev = &unicam->pdev->dev;
> +
> + ret = vb2_queue_init(q);
> + if (ret) {
> + unicam_err(unicam, "vb2_queue_init() failed\n");
> + return ret;
> + }
> +
> + INIT_LIST_HEAD(&node->dma_queue);
> +
> + vdev->release = unicam_node_release;
> + vdev->fops = &unicam_fops;
> + vdev->ioctl_ops = &unicam_ioctl_ops;
> + vdev->v4l2_dev = &unicam->v4l2_dev;
> + vdev->vfl_dir = VFL_DIR_RX;
> + vdev->queue = q;
> + vdev->lock = &node->lock;
> + vdev->device_caps = (pad_id == IMAGE_PAD) ?
> + (V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING) :
> + (V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING);
> +
> + /* Define the device names */
> + snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
> + pad_id == IMAGE_PAD ? "image" : "embedded");
> +
> + video_set_drvdata(vdev, node);
> + if (pad_id == IMAGE_PAD)
> + vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> + node->pad.flags = MEDIA_PAD_FL_SINK;
> + media_entity_pads_init(&vdev->entity, 1, &node->pad);
> +
> + node->dummy_buf_cpu_addr = dma_alloc_coherent(&unicam->pdev->dev,
> + DUMMY_BUF_SIZE,
> + &node->dummy_buf_dma_addr,
> + GFP_KERNEL);
> + if (!node->dummy_buf_cpu_addr) {
> + unicam_err(unicam, "Unable to allocate dummy buffer.\n");
> + return -ENOMEM;
> + }
> +
> + if (pad_id == METADATA_PAD) {
> + v4l2_disable_ioctl(vdev, VIDIOC_DQEVENT);
> + v4l2_disable_ioctl(vdev, VIDIOC_SUBSCRIBE_EVENT);
> + v4l2_disable_ioctl(vdev, VIDIOC_UNSUBSCRIBE_EVENT);
Why?
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_STD);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_STD);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUMSTD);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, querystd))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_QUERYSTD);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_dv_timings)) {
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_EDID);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_EDID);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_DV_TIMINGS_CAP);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUM_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_QUERY_DV_TIMINGS);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, enum_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev,
> + VIDIOC_ENUM_FRAMEINTERVALS);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, g_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_PARM);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_PARM);
> +
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, enum_frame_size))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUM_FRAMESIZES);
> +
> + if (node->pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, set_selection))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_SELECTION);
> +
> + if (node->pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, get_selection))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_SELECTION);
Some of these are probably no longer needed since the v4l2 core will disable
them for you.
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + unicam_err(unicam, "Unable to register video device %s\n",
> + vdev->name);
> + return ret;
> + }
> +
> + /*
> + * Acquire a reference to unicam, which will be released when the video
> + * device will be unregistered and userspace will have closed all open
> + * file handles.
> + */
> + unicam_get(unicam);
> + node->registered = true;
> +
> + if (pad_id != METADATA_PAD || unicam->sensor_embedded_data) {
> + ret = media_create_pad_link(&unicam->sensor->entity, pad_id,
> + &node->video_dev.entity, 0,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> + if (ret)
> + unicam_err(unicam, "Unable to create pad link for %s\n",
> + vdev->name);
> + }
> +
> + return ret;
> +}
> +
> +static void unregister_nodes(struct unicam_device *unicam)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + struct unicam_node *node = &unicam->node[i];
> +
> + if (node->dummy_buf_cpu_addr) {
> + dma_free_coherent(&unicam->pdev->dev, DUMMY_BUF_SIZE,
> + node->dummy_buf_cpu_addr,
> + node->dummy_buf_dma_addr);
> + }
> +
> + if (node->registered) {
> + node->registered = false;
> + video_unregister_device(&node->video_dev);
> + }
> + }
> +}
> +
> +static int unicam_probe_complete(struct unicam_device *unicam)
> +{
> + int ret;
> +
> + unicam->v4l2_dev.notify = unicam_notify;
> +
> + unicam->sensor_config = v4l2_subdev_alloc_pad_config(unicam->sensor);
> + if (!unicam->sensor_config)
> + return -ENOMEM;
> +
> + unicam->sensor_embedded_data = (unicam->sensor->entity.num_pads >= 2);
> +
> + ret = register_node(unicam, &unicam->node[IMAGE_PAD],
> + V4L2_BUF_TYPE_VIDEO_CAPTURE, IMAGE_PAD);
> + if (ret) {
> + unicam_err(unicam, "Unable to register image video device.\n");
> + goto unregister;
> + }
> +
> + ret = register_node(unicam, &unicam->node[METADATA_PAD],
> + V4L2_BUF_TYPE_META_CAPTURE, METADATA_PAD);
> + if (ret) {
> + unicam_err(unicam, "Unable to register metadata video device.\n");
> + goto unregister;
> + }
> +
> + ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
> + if (ret) {
> + unicam_err(unicam, "Unable to register subdev nodes.\n");
> + goto unregister;
> + }
> +
> + /*
> + * Release the initial reference, all references are now owned by the
> + * video devices.
> + */
> + unicam_put(unicam);
> + return 0;
> +
> +unregister:
> + unregister_nodes(unicam);
> + unicam_put(unicam);
> +
> + return ret;
> +}
> +
> +static int unicam_async_complete(struct v4l2_async_notifier *notifier)
> +{
> + struct unicam_device *unicam = to_unicam_device(notifier->v4l2_dev);
> +
> + return unicam_probe_complete(unicam);
> +}
> +
> +static const struct v4l2_async_notifier_operations unicam_async_ops = {
> + .bound = unicam_async_bound,
> + .complete = unicam_async_complete,
> +};
> +
> +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> +{
> + struct platform_device *pdev = dev->pdev;
> + struct v4l2_fwnode_endpoint ep = { 0 };
> + struct device_node *ep_node;
> + struct device_node *sensor_node;
> + unsigned int lane;
> + int ret = -EINVAL;
> +
> + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> + &dev->max_data_lanes) < 0) {
> + unicam_err(dev, "number of data lanes not set\n");
> + return -EINVAL;
> + }
> +
> + /* Get the local endpoint and remote device. */
> + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> + if (!ep_node) {
> + unicam_dbg(3, dev, "can't get next endpoint\n");
> + return -EINVAL;
> + }
> +
> + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> +
> + sensor_node = of_graph_get_remote_port_parent(ep_node);
> + if (!sensor_node) {
> + unicam_dbg(3, dev, "can't get remote parent\n");
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> +
> + /* Parse the local endpoint and validate its configuration. */
> + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> +
> + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> + ep.bus_type);
> +
> + dev->bus_type = ep.bus_type;
> +
> + switch (ep.bus_type) {
> + case V4L2_MBUS_CSI2_DPHY:
> + switch (ep.bus.mipi_csi2.num_data_lanes) {
> + case 1:
> + case 2:
> + case 4:
> + break;
> +
> + default:
> + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> + sensor_node,
> + ep.bus.mipi_csi2.num_data_lanes);
> + goto cleanup_exit;
> + }
> +
> + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> + sensor_node);
> + goto cleanup_exit;
> + }
> + }
> +
> + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> + ep.bus.mipi_csi2.num_data_lanes,
> + dev->max_data_lanes);
> + }
> +
> + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> + dev->bus_flags = ep.bus.mipi_csi2.flags;
> +
> + break;
> +
> + case V4L2_MBUS_CCP2:
> + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> + ep.bus.mipi_csi1.data_lane != 1) {
> + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> + sensor_node);
> + goto cleanup_exit;
> + }
> +
> + dev->max_data_lanes = 1;
> + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> + break;
> +
> + default:
> + /* Unsupported bus type */
> + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> + sensor_node, ep.bus_type);
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> + sensor_node,
> + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> + dev->max_data_lanes, dev->bus_flags);
> +
> + /* Initialize and register the async notifier. */
> + v4l2_async_notifier_init(&dev->notifier);
> + dev->notifier.ops = &unicam_async_ops;
> +
> + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> + if (ret) {
> + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> + goto cleanup_exit;
> + }
> +
> + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> + if (ret) {
> + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> + ret = -EINVAL;
> + }
> +
> +cleanup_exit:
> + of_node_put(sensor_node);
> + of_node_put(ep_node);
> +
> + return ret;
> +}
> +
> +static int unicam_probe(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam;
> + int ret;
> +
> + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> + if (!unicam)
> + return -ENOMEM;
> +
> + kref_init(&unicam->kref);
> + unicam->pdev = pdev;
> +
> + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(unicam->base)) {
> + unicam_err(unicam, "Failed to get main io block\n");
> + ret = PTR_ERR(unicam->base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> + if (IS_ERR(unicam->clk_gate_base)) {
> + unicam_err(unicam, "Failed to get 2nd io block\n");
> + ret = PTR_ERR(unicam->clk_gate_base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> + if (IS_ERR(unicam->clock)) {
> + unicam_err(unicam, "Failed to get clock\n");
> + ret = PTR_ERR(unicam->clock);
> + goto err_unicam_put;
> + }
> +
> + ret = platform_get_irq(pdev, 0);
> + if (ret <= 0) {
> + dev_err(&pdev->dev, "No IRQ resource\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> + "unicam_capture0", unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Unable to request interrupt\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + unicam->mdev.dev = &pdev->dev;
> + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> + sizeof(unicam->mdev.model));
> + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> + "platform:%s", dev_name(&pdev->dev));
> + unicam->mdev.hw_revision = 0;
> +
> + media_device_init(&unicam->mdev);
> +
> + unicam->v4l2_dev.mdev = &unicam->mdev;
> +
> + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> + if (ret) {
> + unicam_err(unicam,
> + "Unable to register v4l2 device.\n");
> + goto err_unicam_put;
> + }
> +
> + ret = media_device_register(&unicam->mdev);
> + if (ret < 0) {
> + unicam_err(unicam,
> + "Unable to register media-controller device.\n");
> + goto err_v4l2_unregister;
> + }
> +
> + /* Reserve space for the controls */
> + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> + if (ret < 0)
> + goto err_media_unregister;
> +
> + /* set the driver data in platform device */
> + platform_set_drvdata(pdev, unicam);
> +
> + ret = of_unicam_connect_subdevs(unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> + goto err_media_unregister;
> + }
> +
> + /* Enable the block power domain */
> + pm_runtime_enable(&pdev->dev);
> +
> + return 0;
> +
> +err_media_unregister:
> + media_device_unregister(&unicam->mdev);
> +err_v4l2_unregister:
> + v4l2_device_unregister(&unicam->v4l2_dev);
> +err_unicam_put:
> + unicam_put(unicam);
> +
> + return ret;
> +}
> +
> +static int unicam_remove(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam = platform_get_drvdata(pdev);
> +
> + unicam_dbg(2, unicam, "%s\n", __func__);
> +
> + v4l2_async_notifier_unregister(&unicam->notifier);
> + v4l2_device_unregister(&unicam->v4l2_dev);
> + media_device_unregister(&unicam->mdev);
> + unregister_nodes(unicam);
> +
> + pm_runtime_disable(&pdev->dev);
> +
> + return 0;
> +}
> +
> +static const struct of_device_id unicam_of_match[] = {
> + { .compatible = "brcm,bcm2835-unicam", },
> + { /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> +
> +static struct platform_driver unicam_driver = {
> + .probe = unicam_probe,
> + .remove = unicam_remove,
> + .driver = {
> + .name = UNICAM_MODULE_NAME,
> + .of_match_table = of_match_ptr(unicam_of_match),
> + },
> +};
> +
> +module_platform_driver(unicam_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> +MODULE_LICENSE("GPL");
> +MODULE_VERSION(UNICAM_VERSION);
> diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> new file mode 100644
> index 000000000000..ae059a171d0f
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> @@ -0,0 +1,253 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +/*
> + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + */
> +
> +#ifndef VC4_REGS_UNICAM_H
> +#define VC4_REGS_UNICAM_H
> +
> +/*
> + * The following values are taken from files found within the code drop
> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> + * They have been modified to be only the register offset.
> + */
> +#define UNICAM_CTRL 0x000
> +#define UNICAM_STA 0x004
> +#define UNICAM_ANA 0x008
> +#define UNICAM_PRI 0x00c
> +#define UNICAM_CLK 0x010
> +#define UNICAM_CLT 0x014
> +#define UNICAM_DAT0 0x018
> +#define UNICAM_DAT1 0x01c
> +#define UNICAM_DAT2 0x020
> +#define UNICAM_DAT3 0x024
> +#define UNICAM_DLT 0x028
> +#define UNICAM_CMP0 0x02c
> +#define UNICAM_CMP1 0x030
> +#define UNICAM_CAP0 0x034
> +#define UNICAM_CAP1 0x038
> +#define UNICAM_ICTL 0x100
> +#define UNICAM_ISTA 0x104
> +#define UNICAM_IDI0 0x108
> +#define UNICAM_IPIPE 0x10c
> +#define UNICAM_IBSA0 0x110
> +#define UNICAM_IBEA0 0x114
> +#define UNICAM_IBLS 0x118
> +#define UNICAM_IBWP 0x11c
> +#define UNICAM_IHWIN 0x120
> +#define UNICAM_IHSTA 0x124
> +#define UNICAM_IVWIN 0x128
> +#define UNICAM_IVSTA 0x12c
> +#define UNICAM_ICC 0x130
> +#define UNICAM_ICS 0x134
> +#define UNICAM_IDC 0x138
> +#define UNICAM_IDPO 0x13c
> +#define UNICAM_IDCA 0x140
> +#define UNICAM_IDCD 0x144
> +#define UNICAM_IDS 0x148
> +#define UNICAM_DCS 0x200
> +#define UNICAM_DBSA0 0x204
> +#define UNICAM_DBEA0 0x208
> +#define UNICAM_DBWP 0x20c
> +#define UNICAM_DBCTL 0x300
> +#define UNICAM_IBSA1 0x304
> +#define UNICAM_IBEA1 0x308
> +#define UNICAM_IDI1 0x30c
> +#define UNICAM_DBSA1 0x310
> +#define UNICAM_DBEA1 0x314
> +#define UNICAM_MISC 0x400
> +
> +/*
> + * The following bitmasks are from the kernel released by Broadcom
> + * for Android - https://android.googlesource.com/kernel/bcm/
> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> + * Unicam block as BCM2835, as defined in eg
> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> + * Values reworked to use the kernel BIT and GENMASK macros.
> + *
> + * Some of the bit mnenomics have been amended to match the datasheet.
> + */
> +/* UNICAM_CTRL Register */
> +#define UNICAM_CPE BIT(0)
> +#define UNICAM_MEM BIT(1)
> +#define UNICAM_CPR BIT(2)
> +#define UNICAM_CPM_MASK GENMASK(3, 3)
> +#define UNICAM_CPM_CSI2 0
> +#define UNICAM_CPM_CCP2 1
> +#define UNICAM_SOE BIT(4)
> +#define UNICAM_DCM_MASK GENMASK(5, 5)
> +#define UNICAM_DCM_STROBE 0
> +#define UNICAM_DCM_DATA 1
> +#define UNICAM_SLS BIT(6)
> +#define UNICAM_PFT_MASK GENMASK(11, 8)
> +#define UNICAM_OET_MASK GENMASK(20, 12)
> +
> +/* UNICAM_STA Register */
> +#define UNICAM_SYN BIT(0)
> +#define UNICAM_CS BIT(1)
> +#define UNICAM_SBE BIT(2)
> +#define UNICAM_PBE BIT(3)
> +#define UNICAM_HOE BIT(4)
> +#define UNICAM_PLE BIT(5)
> +#define UNICAM_SSC BIT(6)
> +#define UNICAM_CRCE BIT(7)
> +#define UNICAM_OES BIT(8)
> +#define UNICAM_IFO BIT(9)
> +#define UNICAM_OFO BIT(10)
> +#define UNICAM_BFO BIT(11)
> +#define UNICAM_DL BIT(12)
> +#define UNICAM_PS BIT(13)
> +#define UNICAM_IS BIT(14)
> +#define UNICAM_PI0 BIT(15)
> +#define UNICAM_PI1 BIT(16)
> +#define UNICAM_FSI_S BIT(17)
> +#define UNICAM_FEI_S BIT(18)
> +#define UNICAM_LCI_S BIT(19)
> +#define UNICAM_BUF0_RDY BIT(20)
> +#define UNICAM_BUF0_NO BIT(21)
> +#define UNICAM_BUF1_RDY BIT(22)
> +#define UNICAM_BUF1_NO BIT(23)
> +#define UNICAM_DI BIT(24)
> +
> +#define UNICAM_STA_MASK_ALL \
> + (UNICAM_DL + \
> + UNICAM_SBE + \
> + UNICAM_PBE + \
> + UNICAM_HOE + \
> + UNICAM_PLE + \
> + UNICAM_SSC + \
> + UNICAM_CRCE + \
> + UNICAM_IFO + \
> + UNICAM_OFO + \
> + UNICAM_PS + \
> + UNICAM_PI0 + \
> + UNICAM_PI1)
> +
> +/* UNICAM_ANA Register */
> +#define UNICAM_APD BIT(0)
> +#define UNICAM_BPD BIT(1)
> +#define UNICAM_AR BIT(2)
> +#define UNICAM_DDL BIT(3)
> +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> +
> +/* UNICAM_PRI Register */
> +#define UNICAM_PE BIT(0)
> +#define UNICAM_PT_MASK GENMASK(2, 1)
> +#define UNICAM_NP_MASK GENMASK(7, 4)
> +#define UNICAM_PP_MASK GENMASK(11, 8)
> +#define UNICAM_BS_MASK GENMASK(15, 12)
> +#define UNICAM_BL_MASK GENMASK(17, 16)
> +
> +/* UNICAM_CLK Register */
> +#define UNICAM_CLE BIT(0)
> +#define UNICAM_CLPD BIT(1)
> +#define UNICAM_CLLPE BIT(2)
> +#define UNICAM_CLHSE BIT(3)
> +#define UNICAM_CLTRE BIT(4)
> +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> +#define UNICAM_CLSTE BIT(29)
> +
> +/* UNICAM_CLT Register */
> +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DATn Registers */
> +#define UNICAM_DLE BIT(0)
> +#define UNICAM_DLPD BIT(1)
> +#define UNICAM_DLLPE BIT(2)
> +#define UNICAM_DLHSE BIT(3)
> +#define UNICAM_DLTRE BIT(4)
> +#define UNICAM_DLSM BIT(5)
> +#define UNICAM_DLFO BIT(28)
> +#define UNICAM_DLSTE BIT(29)
> +
> +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> +
> +/* UNICAM_DLT Register */
> +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> +
> +/* UNICAM_ICTL Register */
> +#define UNICAM_FSIE BIT(0)
> +#define UNICAM_FEIE BIT(1)
> +#define UNICAM_IBOB BIT(2)
> +#define UNICAM_FCM BIT(3)
> +#define UNICAM_TFC BIT(4)
> +#define UNICAM_LIP_MASK GENMASK(6, 5)
> +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> +
> +/* UNICAM_IDI0/1 Register */
> +#define UNICAM_ID0_MASK GENMASK(7, 0)
> +#define UNICAM_ID1_MASK GENMASK(15, 8)
> +#define UNICAM_ID2_MASK GENMASK(23, 16)
> +#define UNICAM_ID3_MASK GENMASK(31, 24)
> +
> +/* UNICAM_ISTA Register */
> +#define UNICAM_FSI BIT(0)
> +#define UNICAM_FEI BIT(1)
> +#define UNICAM_LCI BIT(2)
> +
> +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> +
> +/* UNICAM_IPIPE Register */
> +#define UNICAM_PUM_MASK GENMASK(2, 0)
> + /* Unpacking modes */
> + #define UNICAM_PUM_NONE 0
> + #define UNICAM_PUM_UNPACK6 1
> + #define UNICAM_PUM_UNPACK7 2
> + #define UNICAM_PUM_UNPACK8 3
> + #define UNICAM_PUM_UNPACK10 4
> + #define UNICAM_PUM_UNPACK12 5
> + #define UNICAM_PUM_UNPACK14 6
> + #define UNICAM_PUM_UNPACK16 7
> +#define UNICAM_DDM_MASK GENMASK(6, 3)
> +#define UNICAM_PPM_MASK GENMASK(9, 7)
> + /* Packing modes */
> + #define UNICAM_PPM_NONE 0
> + #define UNICAM_PPM_PACK8 1
> + #define UNICAM_PPM_PACK10 2
> + #define UNICAM_PPM_PACK12 3
> + #define UNICAM_PPM_PACK14 4
> + #define UNICAM_PPM_PACK16 5
> +#define UNICAM_DEM_MASK GENMASK(11, 10)
> +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> +#define UNICAM_ICM_MASK GENMASK(16, 15)
> +#define UNICAM_IDM_MASK GENMASK(17, 17)
> +
> +/* UNICAM_ICC Register */
> +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> +#define UNICAM_ICST_MASK GENMASK(12, 10)
> +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> +
> +/* UNICAM_DCS Register */
> +#define UNICAM_DIE BIT(0)
> +#define UNICAM_DIM BIT(1)
> +#define UNICAM_DBOB BIT(3)
> +#define UNICAM_FDE BIT(4)
> +#define UNICAM_LDP BIT(5)
> +#define UNICAM_EDL_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DBCTL Register */
> +#define UNICAM_DBEN BIT(0)
> +#define UNICAM_BUF0_IE BIT(1)
> +#define UNICAM_BUF1_IE BIT(2)
> +
> +/* UNICAM_CMP[0,1] register */
> +#define UNICAM_PCE BIT(31)
> +#define UNICAM_GI BIT(9)
> +#define UNICAM_CPH BIT(8)
> +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> +
> +/* UNICAM_MISC register */
> +#define UNICAM_FL0 BIT(6)
> +#define UNICAM_FL1 BIT(9)
> +
> +#endif
>
Regards,
Hans
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-05-04 9:25 ` [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface Laurent Pinchart
@ 2020-05-05 1:26 ` kbuild test robot
2020-05-05 1:26 ` kbuild test robot
` (3 subsequent siblings)
4 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 1:26 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: kbuild-all, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 5160 bytes --]
Hi Laurent,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200504]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
config: c6x-allyesconfig (attached as .config)
compiler: c6x-elf-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=c6x
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
All error/warnings (new ones prefixed by >>):
drivers/media/platform/bcm2835/bcm2835-unicam.c: In function 'unicam_probe_complete':
>> drivers/media/platform/bcm2835/bcm2835-unicam.c:2531:8: error: implicit declaration of function 'v4l2_device_register_ro_subdev_nodes'; did you mean 'v4l2_device_register_subdev_nodes'? [-Werror=implicit-function-declaration]
2531 | ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| v4l2_device_register_subdev_nodes
drivers/media/platform/bcm2835/bcm2835-unicam.c: In function 'get_first_supported_format':
>> drivers/media/platform/bcm2835/bcm2835-unicam.c:1014:18: warning: 'ret' is used uninitialized in this function [-Wuninitialized]
1014 | for (i = 0; ret != -EINVAL && ret != -ENOIOCTLCMD; ++i) {
during RTL pass: mach
drivers/media/platform/bcm2835/bcm2835-unicam.c: In function 'register_node':
drivers/media/platform/bcm2835/bcm2835-unicam.c:2483:1: internal compiler error: in priority, at haifa-sched.c:1599
2483 | }
| ^
0x527435 priority
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:1599
0xeeff2a priority
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:1595
0xeeff2a apply_replacement
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:4719
0xef0b20 undo_replacements_for_backtrack
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:4454
0xef0d63 restore_last_backtrack_point
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:4551
0xef4f5e schedule_block(basic_block_def**, void*)
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:6952
0xf72558 schedule_ebb(rtx_insn*, rtx_insn*, bool)
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/sched-ebb.c:537
0xf72bd1 schedule_ebbs()
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/sched-ebb.c:656
0xc94f9b c6x_reorg
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/config/c6x/c6x.c:5943
0x984d9d execute
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/reorg.c:3992
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
vim +2531 drivers/media/platform/bcm2835/bcm2835-unicam.c
2504
2505 static int unicam_probe_complete(struct unicam_device *unicam)
2506 {
2507 int ret;
2508
2509 unicam->v4l2_dev.notify = unicam_notify;
2510
2511 unicam->sensor_config = v4l2_subdev_alloc_pad_config(unicam->sensor);
2512 if (!unicam->sensor_config)
2513 return -ENOMEM;
2514
2515 unicam->sensor_embedded_data = (unicam->sensor->entity.num_pads >= 2);
2516
2517 ret = register_node(unicam, &unicam->node[IMAGE_PAD],
2518 V4L2_BUF_TYPE_VIDEO_CAPTURE, IMAGE_PAD);
2519 if (ret) {
2520 unicam_err(unicam, "Unable to register image video device.\n");
2521 goto unregister;
2522 }
2523
2524 ret = register_node(unicam, &unicam->node[METADATA_PAD],
2525 V4L2_BUF_TYPE_META_CAPTURE, METADATA_PAD);
2526 if (ret) {
2527 unicam_err(unicam, "Unable to register metadata video device.\n");
2528 goto unregister;
2529 }
2530
> 2531 ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
2532 if (ret) {
2533 unicam_err(unicam, "Unable to register subdev nodes.\n");
2534 goto unregister;
2535 }
2536
2537 /*
2538 * Release the initial reference, all references are now owned by the
2539 * video devices.
2540 */
2541 unicam_put(unicam);
2542 return 0;
2543
2544 unregister:
2545 unregister_nodes(unicam);
2546 unicam_put(unicam);
2547
2548 return ret;
2549 }
2550
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 52564 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
@ 2020-05-05 1:26 ` kbuild test robot
0 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 1:26 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 5279 bytes --]
Hi Laurent,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200504]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
config: c6x-allyesconfig (attached as .config)
compiler: c6x-elf-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=c6x
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
All error/warnings (new ones prefixed by >>):
drivers/media/platform/bcm2835/bcm2835-unicam.c: In function 'unicam_probe_complete':
>> drivers/media/platform/bcm2835/bcm2835-unicam.c:2531:8: error: implicit declaration of function 'v4l2_device_register_ro_subdev_nodes'; did you mean 'v4l2_device_register_subdev_nodes'? [-Werror=implicit-function-declaration]
2531 | ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| v4l2_device_register_subdev_nodes
drivers/media/platform/bcm2835/bcm2835-unicam.c: In function 'get_first_supported_format':
>> drivers/media/platform/bcm2835/bcm2835-unicam.c:1014:18: warning: 'ret' is used uninitialized in this function [-Wuninitialized]
1014 | for (i = 0; ret != -EINVAL && ret != -ENOIOCTLCMD; ++i) {
during RTL pass: mach
drivers/media/platform/bcm2835/bcm2835-unicam.c: In function 'register_node':
drivers/media/platform/bcm2835/bcm2835-unicam.c:2483:1: internal compiler error: in priority, at haifa-sched.c:1599
2483 | }
| ^
0x527435 priority
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:1599
0xeeff2a priority
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:1595
0xeeff2a apply_replacement
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:4719
0xef0b20 undo_replacements_for_backtrack
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:4454
0xef0d63 restore_last_backtrack_point
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:4551
0xef4f5e schedule_block(basic_block_def**, void*)
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/haifa-sched.c:6952
0xf72558 schedule_ebb(rtx_insn*, rtx_insn*, bool)
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/sched-ebb.c:537
0xf72bd1 schedule_ebbs()
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/sched-ebb.c:656
0xc94f9b c6x_reorg
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/config/c6x/c6x.c:5943
0x984d9d execute
/tmp/build-crosstools-xh-9.3.0-2.34/gcc/gcc-9.3.0/gcc/reorg.c:3992
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
vim +2531 drivers/media/platform/bcm2835/bcm2835-unicam.c
2504
2505 static int unicam_probe_complete(struct unicam_device *unicam)
2506 {
2507 int ret;
2508
2509 unicam->v4l2_dev.notify = unicam_notify;
2510
2511 unicam->sensor_config = v4l2_subdev_alloc_pad_config(unicam->sensor);
2512 if (!unicam->sensor_config)
2513 return -ENOMEM;
2514
2515 unicam->sensor_embedded_data = (unicam->sensor->entity.num_pads >= 2);
2516
2517 ret = register_node(unicam, &unicam->node[IMAGE_PAD],
2518 V4L2_BUF_TYPE_VIDEO_CAPTURE, IMAGE_PAD);
2519 if (ret) {
2520 unicam_err(unicam, "Unable to register image video device.\n");
2521 goto unregister;
2522 }
2523
2524 ret = register_node(unicam, &unicam->node[METADATA_PAD],
2525 V4L2_BUF_TYPE_META_CAPTURE, METADATA_PAD);
2526 if (ret) {
2527 unicam_err(unicam, "Unable to register metadata video device.\n");
2528 goto unregister;
2529 }
2530
> 2531 ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
2532 if (ret) {
2533 unicam_err(unicam, "Unable to register subdev nodes.\n");
2534 goto unregister;
2535 }
2536
2537 /*
2538 * Release the initial reference, all references are now owned by the
2539 * video devices.
2540 */
2541 unicam_put(unicam);
2542 return 0;
2543
2544 unregister:
2545 unregister_nodes(unicam);
2546 unicam_put(unicam);
2547
2548 return ret;
2549 }
2550
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 52564 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-05-04 9:25 ` [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface Laurent Pinchart
2020-05-04 15:21 ` Hans Verkuil
2020-05-05 1:26 ` kbuild test robot
@ 2020-05-06 18:01 ` Nicolas Saenz Julienne
2020-08-29 11:20 ` Jacopo Mondi
2020-09-15 7:03 ` Sakari Ailus
4 siblings, 0 replies; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-06 18:01 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Laurent,
On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Add a driver for the Unicam camera receiver block on BCM283x
> processors.
Nonetheless I've only seen DT patches for bcm2711. Does this also
aplies for rpi[1-3]? And if so, wouldn't it make sense to get rid of
bcm2835-camera?
Regards,
Nicolas
> Compared to the bcm2835-camera driver present in staging, this driver
> handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> the VC4 firmware running on the VPU.
>
> The commit is made up of a series of changes cherry-picked from the
> rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> additional enhancements, forward-ported to the mainline kernel.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v1:
>
> - Re-fetch mbus code from subdev on a g_fmt call
> - Group all ioctl disabling together
> - Fix reference counting in unicam_open
> - Add support for VIDIOC_[S|G]_SELECTION
> ---
> MAINTAINERS | 7 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/bcm2835/Kconfig | 15 +
> drivers/media/platform/bcm2835/Makefile | 3 +
> .../media/platform/bcm2835/bcm2835-unicam.c | 2825
> +++++++++++++++++
> .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> 7 files changed, 3106 insertions(+)
> create mode 100644 drivers/media/platform/bcm2835/Kconfig
> create mode 100644 drivers/media/platform/bcm2835/Makefile
> create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b816a453b10e..edf5b8d9c98a 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3341,6 +3341,13 @@ N: bcm113*
> N: bcm216*
> N: kona
>
> +BROADCOM BCM2835 CAMERA DRIVER
> +M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> +L: linux-media@vger.kernel.org
> +S: Maintained
> +F: drivers/media/platform/bcm2835/
> +F: Documentation/devicetree/bindings/media/brcm,bcm2835-
> unicam.yaml
> +
> BROADCOM BCM47XX MIPS ARCHITECTURE
> M: Hauke Mehrtens <hauke@hauke-m.de>
> M: Rafał Miłecki <zajec5@gmail.com>
> diff --git a/drivers/media/platform/Kconfig
> b/drivers/media/platform/Kconfig
> index e01bbb9dd1c1..98721a4e0be1 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -146,6 +146,7 @@ source "drivers/media/platform/am437x/Kconfig"
> source "drivers/media/platform/xilinx/Kconfig"
> source "drivers/media/platform/rcar-vin/Kconfig"
> source "drivers/media/platform/atmel/Kconfig"
> +source "drivers/media/platform/bcm2835/Kconfig"
> source "drivers/media/platform/sunxi/Kconfig"
>
> config VIDEO_TI_CAL
> diff --git a/drivers/media/platform/Makefile
> b/drivers/media/platform/Makefile
> index d13db96e3015..a425e4d2e3f3 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -98,4 +98,6 @@ obj-y +=
> meson/
>
> obj-y += cros-ec-cec/
>
> +obj-y += bcm2835/
> +
> obj-y += sunxi/
> diff --git a/drivers/media/platform/bcm2835/Kconfig
> b/drivers/media/platform/bcm2835/Kconfig
> new file mode 100644
> index 000000000000..ec46e3ef053c
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/Kconfig
> @@ -0,0 +1,15 @@
> +# Broadcom VideoCore4 V4L2 camera support
> +
> +config VIDEO_BCM2835_UNICAM
> + tristate "Broadcom BCM2835 Unicam video capture driver"
> + depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API &&
> MEDIA_CONTROLLER
> + depends on ARCH_BCM2835 || COMPILE_TEST
> + select VIDEOBUF2_DMA_CONTIG
> + select V4L2_FWNODE
> + help
> + Say Y here to enable support for the BCM2835 CSI-2 receiver.
> This is a
> + V4L2 driver that controls the CSI-2 receiver directly,
> independently
> + from the VC4 firmware.
> +
> + To compile this driver as a module, choose M here. The module
> will be
> + called bcm2835-unicam.
> diff --git a/drivers/media/platform/bcm2835/Makefile
> b/drivers/media/platform/bcm2835/Makefile
> new file mode 100644
> index 000000000000..a98aba03598a
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/Makefile
> @@ -0,0 +1,3 @@
> +# Makefile for BCM2835 Unicam driver
> +
> +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c
> b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> new file mode 100644
> index 000000000000..2e9387cbc1e0
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> @@ -0,0 +1,2825 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * BCM2835 Unicam Capture Driver
> + *
> + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> + *
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + *
> + * Based on TI am437x driver by
> + * Benoit Parrot <bparrot@ti.com>
> + * Lad, Prabhakar <prabhakar.csengg@gmail.com>
> + *
> + * and TI CAL camera interface driver by
> + * Benoit Parrot <bparrot@ti.com>
> + *
> + *
> + * There are two camera drivers in the kernel for BCM283x - this one
> + * and bcm2835-camera (currently in staging).
> + *
> + * This driver directly controls the Unicam peripheral - there is no
> + * involvement with the VideoCore firmware. Unicam receives CSI-2 or
> + * CCP2 data and writes it into SDRAM.
> + * The only potential processing options are to repack Bayer data
> into an
> + * alternate format, and applying windowing.
> + * The repacking does not shift the data, so can repack
> V4L2_PIX_FMT_Sxxxx10P
> + * to V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to
> V4L2_PIX_FMT_Sxxxx12,
> + * but not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will
> add both
> + * formats where the relevant formats are defined, and will
> automatically
> + * configure the repacking as required.
> + * Support for windowing may be added later.
> + *
> + * It should be possible to connect this driver to any sensor with a
> + * suitable output interface and V4L2 subdevice driver.
> + *
> + * bcm2835-camera uses the VideoCore firmware to control the sensor,
> + * Unicam, ISP, and all tuner control loops. Fully processed frames
> are
> + * delivered to the driver by the firmware. It only has sensor
> drivers
> + * for Omnivision OV5647, and Sony IMX219 sensors.
> + *
> + * The two drivers are mutually exclusive for the same Unicam
> instance.
> + * The VideoCore firmware checks the device tree configuration
> during boot.
> + * If it finds device tree nodes called csi0 or csi1 it will block
> the
> + * firmware from accessing the peripheral, and bcm2835-camera will
> + * not be able to stream data.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/init.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/of_graph.h>
> +#include <linux/pinctrl/consumer.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> +#include <linux/uaccess.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-dv-timings.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "vc4-regs-unicam.h"
> +
> +#define UNICAM_MODULE_NAME "unicam"
> +#define UNICAM_VERSION "0.1.0"
> +
> +static int debug;
> +module_param(debug, int, 0644);
> +MODULE_PARM_DESC(debug, "Debug level 0-3");
> +
> +#define unicam_dbg(level, dev, fmt, arg...) \
> + v4l2_dbg(level, debug, &(dev)->v4l2_dev, fmt, ##arg)
> +#define unicam_info(dev, fmt, arg...) \
> + v4l2_info(&(dev)->v4l2_dev, fmt, ##arg)
> +#define unicam_err(dev, fmt, arg...) \
> + v4l2_err(&(dev)->v4l2_dev, fmt, ##arg)
> +
> +/*
> + * To protect against a dodgy sensor driver never returning an error
> from
> + * enum_mbus_code, set a maximum index value to be used.
> + */
> +#define MAX_ENUM_MBUS_CODE 128
> +
> +/*
> + * Stride is a 16 bit register, but also has to be a multiple of 32.
> + */
> +#define BPL_ALIGNMENT 32
> +#define MAX_BYTESPERLINE ((1 << 16) - BPL_ALIGNMENT)
> +/*
> + * Max width is therefore determined by the max stride divided by
> + * the number of bits per pixel. Take 32bpp as a
> + * worst case.
> + * No imposed limit on the height, so adopt a square image for want
> + * of anything better.
> + */
> +#define MAX_WIDTH (MAX_BYTESPERLINE / 4)
> +#define MAX_HEIGHT MAX_WIDTH
> +/* Define a nominal minimum image size */
> +#define MIN_WIDTH 16
> +#define MIN_HEIGHT 16
> +/* Default size of the embedded buffer */
> +#define UNICAM_EMBEDDED_SIZE 8192
> +
> +/*
> + * Size of the dummy buffer. Can be any size really, but the DMA
> + * allocation works in units of page sizes.
> + */
> +#define DUMMY_BUF_SIZE (PAGE_SIZE)
> +
> +enum pad_types {
> + IMAGE_PAD,
> + METADATA_PAD,
> + MAX_NODES
> +};
> +
> +/*
> + * struct unicam_fmt - Unicam media bus format information
> + * @pixelformat: V4L2 pixel format FCC identifier. 0 if n/a.
> + * @repacked_fourcc: V4L2 pixel format FCC identifier if the data is
> expanded
> + * out to 16bpp. 0 if n/a.
> + * @code: V4L2 media bus format code.
> + * @depth: Bits per pixel as delivered from the source.
> + * @csi_dt: CSI data type.
> + * @check_variants: Flag to denote that there are multiple mediabus
> formats
> + * still in the list that could match this V4L2 format.
> + */
> +struct unicam_fmt {
> + u32 fourcc;
> + u32 repacked_fourcc;
> + u32 code;
> + u8 depth;
> + u8 csi_dt;
> + u8 check_variants;
> +};
> +
> +static const struct unicam_fmt formats[] = {
> + /* YUV Formats */
> + {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .code = MEDIA_BUS_FMT_YUYV8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .code = MEDIA_BUS_FMT_UYVY8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVYU,
> + .code = MEDIA_BUS_FMT_YVYU8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_VYUY,
> + .code = MEDIA_BUS_FMT_VYUY8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .code = MEDIA_BUS_FMT_YUYV8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVYU,
> + .code = MEDIA_BUS_FMT_YVYU8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_VYUY,
> + .code = MEDIA_BUS_FMT_VYUY8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + /* RGB Formats */
> + .fourcc = V4L2_PIX_FMT_RGB565, /* gggbbbbb
> rrrrrggg */
> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
> + .depth = 16,
> + .csi_dt = 0x22,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB565X, /* rrrrrggg
> gggbbbbb */
> + .code = MEDIA_BUS_FMT_RGB565_2X8_BE,
> + .depth = 16,
> + .csi_dt = 0x22
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB555, /* gggbbbbb
> arrrrrgg */
> + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
> + .depth = 16,
> + .csi_dt = 0x21,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB555X, /* arrrrrgg
> gggbbbbb */
> + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
> + .depth = 16,
> + .csi_dt = 0x21,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB24, /* rgb */
> + .code = MEDIA_BUS_FMT_RGB888_1X24,
> + .depth = 24,
> + .csi_dt = 0x24,
> + }, {
> + .fourcc = V4L2_PIX_FMT_BGR24, /* bgr */
> + .code = MEDIA_BUS_FMT_BGR888_1X24,
> + .depth = 24,
> + .csi_dt = 0x24,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB32, /* argb */
> + .code = MEDIA_BUS_FMT_ARGB8888_1X32,
> + .depth = 32,
> + .csi_dt = 0x0,
> + }, {
> + /* Bayer Formats */
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + .code = MEDIA_BUS_FMT_SBGGR8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + .code = MEDIA_BUS_FMT_SGBRG8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + .code = MEDIA_BUS_FMT_SGRBG8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + .code = MEDIA_BUS_FMT_SRGGB8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> + .code = MEDIA_BUS_FMT_SBGGR12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> + .code = MEDIA_BUS_FMT_SGBRG12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> + .code = MEDIA_BUS_FMT_SGRBG12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> + .code = MEDIA_BUS_FMT_SRGGB12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR14P,
> + .code = MEDIA_BUS_FMT_SBGGR14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG14P,
> + .code = MEDIA_BUS_FMT_SGBRG14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG14P,
> + .code = MEDIA_BUS_FMT_SGRBG14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB14P,
> + .code = MEDIA_BUS_FMT_SRGGB14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + /*
> + * 16 bit Bayer formats could be supported, but there is no
> CSI2
> + * data_type defined for raw 16, and no sensors that produce it
> at
> + * present.
> + */
> +
> + /* Greyscale formats */
> + .fourcc = V4L2_PIX_FMT_GREY,
> + .code = MEDIA_BUS_FMT_Y8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_Y10P,
> + .repacked_fourcc = V4L2_PIX_FMT_Y10,
> + .code = MEDIA_BUS_FMT_Y10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + /* NB There is no packed V4L2 fourcc for this format.
> */
> + .repacked_fourcc = V4L2_PIX_FMT_Y12,
> + .code = MEDIA_BUS_FMT_Y12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + },
> + /* Embedded data format */
> + {
> + .fourcc = V4L2_META_FMT_SENSOR_DATA,
> + .code = MEDIA_BUS_FMT_SENSOR_DATA,
> + .depth = 8,
> + }
> +};
> +
> +struct unicam_buffer {
> + struct vb2_v4l2_buffer vb;
> + struct list_head list;
> +};
> +
> +static inline struct unicam_buffer *to_unicam_buffer(struct
> vb2_buffer *vb)
> +{
> + return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> +}
> +
> +struct unicam_node {
> + bool registered;
> + int open;
> + bool streaming;
> + unsigned int pad_id;
> + /* Pointer pointing to current v4l2_buffer */
> + struct unicam_buffer *cur_frm;
> + /* Pointer pointing to next v4l2_buffer */
> + struct unicam_buffer *next_frm;
> + /* video capture */
> + const struct unicam_fmt *fmt;
> + /* Used to store current pixel format */
> + struct v4l2_format v_fmt;
> + /* Used to store current mbus frame format */
> + struct v4l2_mbus_framefmt m_fmt;
> + /* Buffer queue used in video-buf */
> + struct vb2_queue buffer_queue;
> + /* Queue of filled frames */
> + struct list_head dma_queue;
> + /* IRQ lock for DMA queue */
> + spinlock_t dma_queue_lock;
> + /* lock used to access this structure */
> + struct mutex lock;
> + /* Identifies video device for this channel */
> + struct video_device video_dev;
> + /* Pointer to the parent handle */
> + struct unicam_device *dev;
> + struct media_pad pad;
> + unsigned int embedded_lines;
> + /*
> + * Dummy buffer intended to be used by unicam
> + * if we have no other queued buffers to swap to.
> + */
> + void *dummy_buf_cpu_addr;
> + dma_addr_t dummy_buf_dma_addr;
> +};
> +
> +struct unicam_device {
> + struct kref kref;
> +
> + /* V4l2 specific parameters */
> + struct v4l2_async_subdev asd;
> +
> + /* peripheral base address */
> + void __iomem *base;
> + /* clock gating base address */
> + void __iomem *clk_gate_base;
> + /* clock handle */
> + struct clk *clock;
> + /* V4l2 device */
> + struct v4l2_device v4l2_dev;
> + struct media_device mdev;
> +
> + /* parent device */
> + struct platform_device *pdev;
> + /* subdevice async Notifier */
> + struct v4l2_async_notifier notifier;
> + unsigned int sequence;
> +
> + /* ptr to sub device */
> + struct v4l2_subdev *sensor;
> + /* Pad config for the sensor */
> + struct v4l2_subdev_pad_config *sensor_config;
> +
> + enum v4l2_mbus_type bus_type;
> + /*
> + * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> + * bus.mipi_csi1.strobe for CCP2.
> + */
> + unsigned int bus_flags;
> + unsigned int max_data_lanes;
> + unsigned int active_data_lanes;
> + bool sensor_embedded_data;
> +
> + struct unicam_node node[MAX_NODES];
> + struct v4l2_ctrl_handler ctrl_handler;
> +};
> +
> +static inline struct unicam_device *
> +to_unicam_device(struct v4l2_device *v4l2_dev)
> +{
> + return container_of(v4l2_dev, struct unicam_device, v4l2_dev);
> +}
> +
> +/* Hardware access */
> +static inline void clk_write(struct unicam_device *dev, u32 val)
> +{
> + writel(val | 0x5a000000, dev->clk_gate_base);
> +}
> +
> +static inline u32 reg_read(struct unicam_device *dev, u32 offset)
> +{
> + return readl(dev->base + offset);
> +}
> +
> +static inline void reg_write(struct unicam_device *dev, u32 offset,
> u32 val)
> +{
> + writel(val, dev->base + offset);
> +}
> +
> +static inline int get_field(u32 value, u32 mask)
> +{
> + return (value & mask) >> __ffs(mask);
> +}
> +
> +static inline void set_field(u32 *valp, u32 field, u32 mask)
> +{
> + u32 val = *valp;
> +
> + val &= ~mask;
> + val |= (field << __ffs(mask)) & mask;
> + *valp = val;
> +}
> +
> +static inline u32 reg_read_field(struct unicam_device *dev, u32
> offset,
> + u32 mask)
> +{
> + return get_field(reg_read(dev, offset), mask);
> +}
> +
> +static inline void reg_write_field(struct unicam_device *dev, u32
> offset,
> + u32 field, u32 mask)
> +{
> + u32 val = reg_read(dev, offset);
> +
> + set_field(&val, field, mask);
> + reg_write(dev, offset, val);
> +}
> +
> +/* Power management functions */
> +static inline int unicam_runtime_get(struct unicam_device *dev)
> +{
> + return pm_runtime_get_sync(&dev->pdev->dev);
> +}
> +
> +static inline void unicam_runtime_put(struct unicam_device *dev)
> +{
> + pm_runtime_put_sync(&dev->pdev->dev);
> +}
> +
> +/* Format setup functions */
> +static const struct unicam_fmt *find_format_by_code(u32 code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> + if (formats[i].code == code)
> + return &formats[i];
> + }
> +
> + return NULL;
> +}
> +
> +static int check_mbus_format(struct unicam_device *dev,
> + const struct unicam_fmt *format)
> +{
> + unsigned int i;
> + int ret = 0;
> +
> + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = i,
> + .pad = IMAGE_PAD,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> +
> + ret = v4l2_subdev_call(dev->sensor, pad,
> enum_mbus_code,
> + NULL, &mbus_code);
> +
> + if (!ret && mbus_code.code == format->code)
> + return 1;
> + }
> +
> + return 0;
> +}
> +
> +static const struct unicam_fmt *find_format_by_pix(struct
> unicam_device *dev,
> + u32 pixelformat)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> + if (formats[i].fourcc == pixelformat ||
> + formats[i].repacked_fourcc == pixelformat) {
> + if (formats[i].check_variants &&
> + !check_mbus_format(dev, &formats[i]))
> + continue;
> + return &formats[i];
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static inline unsigned int bytes_per_line(u32 width,
> + const struct unicam_fmt *fmt,
> + u32 v4l2_fourcc)
> +{
> + if (v4l2_fourcc == fmt->repacked_fourcc)
> + /* Repacking always goes to 16bpp */
> + return ALIGN(width << 1, BPL_ALIGNMENT);
> + else
> + return ALIGN((width * fmt->depth) >> 3, BPL_ALIGNMENT);
> +}
> +
> +static int __subdev_get_format(struct unicam_device *dev,
> + struct v4l2_mbus_framefmt *fmt, int
> pad_id)
> +{
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = pad_id
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, get_fmt, dev-
> >sensor_config,
> + &sd_fmt);
> + if (ret < 0)
> + return ret;
> +
> + *fmt = sd_fmt.format;
> +
> + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__,
> + fmt->width, fmt->height, fmt->code);
> +
> + return 0;
> +}
> +
> +static int __subdev_set_format(struct unicam_device *dev,
> + struct v4l2_mbus_framefmt *fmt, int
> pad_id)
> +{
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = pad_id
> + };
> + int ret;
> +
> + sd_fmt.format = *fmt;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev-
> >sensor_config,
> + &sd_fmt);
> + if (ret < 0)
> + return ret;
> +
> + *fmt = sd_fmt.format;
> +
> + if (pad_id == IMAGE_PAD)
> + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__,
> fmt->width,
> + fmt->height, fmt->code);
> + else
> + unicam_dbg(1, dev, "%s Embedded data code:%04x\n",
> __func__,
> + sd_fmt.format.code);
> +
> + return 0;
> +}
> +
> +static int unicam_calc_format_size_bpl(struct unicam_device *dev,
> + const struct unicam_fmt *fmt,
> + struct v4l2_format *f)
> +{
> + unsigned int min_bytesperline;
> +
> + v4l_bound_align_image(&f->fmt.pix.width, MIN_WIDTH, MAX_WIDTH,
> 2,
> + &f->fmt.pix.height, MIN_HEIGHT,
> MAX_HEIGHT, 0,
> + 0);
> +
> + min_bytesperline = bytes_per_line(f->fmt.pix.width, fmt,
> + f->fmt.pix.pixelformat);
> +
> + if (f->fmt.pix.bytesperline > min_bytesperline &&
> + f->fmt.pix.bytesperline <= MAX_BYTESPERLINE)
> + f->fmt.pix.bytesperline = ALIGN(f-
> >fmt.pix.bytesperline,
> + BPL_ALIGNMENT);
> + else
> + f->fmt.pix.bytesperline = min_bytesperline;
> +
> + f->fmt.pix.sizeimage = f->fmt.pix.height * f-
> >fmt.pix.bytesperline;
> +
> + unicam_dbg(3, dev, "%s: fourcc: %08X size: %dx%d bpl:%d
> img_size:%d\n",
> + __func__,
> + f->fmt.pix.pixelformat,
> + f->fmt.pix.width, f->fmt.pix.height,
> + f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
> +
> + return 0;
> +}
> +
> +static int unicam_reset_format(struct unicam_node *node)
> +{
> + struct unicam_device *dev = node->dev;
> + struct v4l2_mbus_framefmt mbus_fmt;
> + int ret;
> +
> + if (dev->sensor_embedded_data || node->pad_id != METADATA_PAD)
> {
> + ret = __subdev_get_format(dev, &mbus_fmt, node-
> >pad_id);
> + if (ret) {
> + unicam_err(dev, "Failed to get_format - ret
> %d\n", ret);
> + return ret;
> + }
> +
> + if (mbus_fmt.code != node->fmt->code) {
> + unicam_err(dev, "code mismatch - fmt->code
> %08x, mbus_fmt.code %08x\n",
> + node->fmt->code, mbus_fmt.code);
> + return ret;
> + }
> + }
> +
> + if (node->pad_id == IMAGE_PAD) {
> + v4l2_fill_pix_format(&node->v_fmt.fmt.pix, &mbus_fmt);
> + node->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + unicam_calc_format_size_bpl(dev, node->fmt, &node-
> >v_fmt);
> + } else {
> + node->v_fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> + node->v_fmt.fmt.meta.dataformat =
> V4L2_META_FMT_SENSOR_DATA;
> + if (dev->sensor_embedded_data) {
> + node->v_fmt.fmt.meta.buffersize =
> + mbus_fmt.width *
> mbus_fmt.height;
> + node->embedded_lines = mbus_fmt.height;
> + } else {
> + node->v_fmt.fmt.meta.buffersize =
> UNICAM_EMBEDDED_SIZE;
> + node->embedded_lines = 1;
> + }
> + }
> +
> + node->m_fmt = mbus_fmt;
> + return 0;
> +}
> +
> +static void unicam_wr_dma_addr(struct unicam_device *dev, dma_addr_t
> dmaaddr,
> + unsigned int buffer_size, int pad_id)
> +{
> + dma_addr_t endaddr = dmaaddr + buffer_size;
> +
> + /*
> + * dmaaddr and endaddr should be a 32-bit address with the top
> two bits
> + * set to 0x3 to signify uncached access through the Videocore
> memory
> + * controller.
> + */
> + WARN_ON((dmaaddr >> 30) != 0x3 || (endaddr >> 30) != 0x3);
> +
> + if (pad_id == IMAGE_PAD) {
> + reg_write(dev, UNICAM_IBSA0, dmaaddr);
> + reg_write(dev, UNICAM_IBEA0, endaddr);
> + } else {
> + reg_write(dev, UNICAM_DBSA0, dmaaddr);
> + reg_write(dev, UNICAM_DBEA0, endaddr);
> + }
> +}
> +
> +static inline unsigned int unicam_get_lines_done(struct
> unicam_device *dev)
> +{
> + dma_addr_t start_addr, cur_addr;
> + unsigned int stride = dev-
> >node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline;
> + struct unicam_buffer *frm = dev->node[IMAGE_PAD].cur_frm;
> +
> + if (!frm)
> + return 0;
> +
> + start_addr = vb2_dma_contig_plane_dma_addr(&frm->vb.vb2_buf,
> 0);
> + cur_addr = reg_read(dev, UNICAM_IBWP);
> + return (unsigned int)(cur_addr - start_addr) / stride;
> +}
> +
> +static inline void unicam_schedule_next_buffer(struct unicam_node
> *node)
> +{
> + struct unicam_device *dev = node->dev;
> + struct unicam_buffer *buf;
> + unsigned int size;
> + dma_addr_t addr;
> +
> + buf = list_first_entry(&node->dma_queue, struct unicam_buffer,
> list);
> + node->next_frm = buf;
> + list_del(&buf->list);
> +
> + addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> + size = (node->pad_id == IMAGE_PAD) ?
> + node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> +
> + unicam_wr_dma_addr(dev, addr, size, node->pad_id);
> +}
> +
> +static inline void unicam_schedule_dummy_buffer(struct unicam_node
> *node)
> +{
> + struct unicam_device *dev = node->dev;
> +
> + unicam_dbg(3, dev, "Scheduling dummy buffer for node %d\n",
> + node->pad_id);
> +
> + unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr,
> DUMMY_BUF_SIZE,
> + node->pad_id);
> + node->next_frm = NULL;
> +}
> +
> +static inline void unicam_process_buffer_complete(struct unicam_node
> *node,
> + unsigned int
> sequence)
> +{
> + node->cur_frm->vb.field = node->m_fmt.field;
> + node->cur_frm->vb.sequence = sequence;
> +
> + vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> VB2_BUF_STATE_DONE);
> +}
> +
> +static bool unicam_all_nodes_streaming(struct unicam_device *dev)
> +{
> + bool ret;
> +
> + ret = dev->node[IMAGE_PAD].open && dev-
> >node[IMAGE_PAD].streaming;
> + ret &= !dev->node[METADATA_PAD].open ||
> + dev->node[METADATA_PAD].streaming;
> + return ret;
> +}
> +
> +static bool unicam_all_nodes_disabled(struct unicam_device *dev)
> +{
> + return !dev->node[IMAGE_PAD].streaming &&
> + !dev->node[METADATA_PAD].streaming;
> +}
> +
> +static void unicam_queue_event_sof(struct unicam_device *unicam)
> +{
> + struct v4l2_event event = {
> + .type = V4L2_EVENT_FRAME_SYNC,
> + .u.frame_sync.frame_sequence = unicam->sequence,
> + };
> +
> + v4l2_event_queue(&unicam->node[IMAGE_PAD].video_dev, &event);
> +}
> +
> +/*
> + * unicam_isr : ISR handler for unicam capture
> + * @irq: irq number
> + * @dev_id: dev_id ptr
> + *
> + * It changes status of the captured buffer, takes next buffer from
> the queue
> + * and sets its address in unicam registers
> + */
> +static irqreturn_t unicam_isr(int irq, void *dev)
> +{
> + struct unicam_device *unicam = dev;
> + unsigned int lines_done = unicam_get_lines_done(dev);
> + unsigned int sequence = unicam->sequence;
> + unsigned int i;
> + u32 ista, sta;
> + u64 ts;
> +
> + /*
> + * Don't service interrupts if not streaming.
> + * Avoids issues if the VPU should enable the
> + * peripheral without the kernel knowing (that
> + * shouldn't happen, but causes issues if it does).
> + */
> + if (unicam_all_nodes_disabled(unicam))
> + return IRQ_NONE;
> +
> + sta = reg_read(unicam, UNICAM_STA);
> + /* Write value back to clear the interrupts */
> + reg_write(unicam, UNICAM_STA, sta);
> +
> + ista = reg_read(unicam, UNICAM_ISTA);
> + /* Write value back to clear the interrupts */
> + reg_write(unicam, UNICAM_ISTA, ista);
> +
> + unicam_dbg(3, unicam, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d,
> lines done %d",
> + ista, sta, sequence, lines_done);
> +
> + if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> + return IRQ_HANDLED;
> +
> + /*
> + * We must run the frame end handler first. If we have a valid
> next_frm
> + * and we get a simultaneout FE + FS interrupt, running the FS
> handler
> + * first would null out the next_frm ptr and we would have lost
> the
> + * buffer forever.
> + */
> + if (ista & UNICAM_FEI || sta & UNICAM_PI0) {
> + /*
> + * Ensure we have swapped buffers already as we can't
> + * stop the peripheral. If no buffer is available, use
> a
> + * dummy buffer to dump out frames until we get a new
> buffer
> + * to use.
> + */
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + if (unicam->node[i].cur_frm)
> + unicam_process_buffer_complete(&unicam-
> >node[i],
> + sequence
> );
> + unicam->node[i].cur_frm = unicam-
> >node[i].next_frm;
> + }
> + unicam->sequence++;
> + }
> +
> + if (ista & UNICAM_FSI) {
> + /*
> + * Timestamp is to be when the first data byte was
> captured,
> + * aka frame start.
> + */
> + ts = ktime_get_ns();
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + if (unicam->node[i].cur_frm)
> + unicam->node[i].cur_frm-
> >vb.vb2_buf.timestamp =
> + ts;
> + /*
> + * Set the next frame output to go to a dummy
> frame
> + * if we have not managed to obtain another
> frame
> + * from the queue.
> + */
> + unicam_schedule_dummy_buffer(&unicam->node[i]);
> + }
> +
> + unicam_queue_event_sof(unicam);
> + }
> +
> + /*
> + * Cannot swap buffer at frame end, there may be a race
> condition
> + * where the HW does not actually swap it if the new frame has
> + * already started.
> + */
> + if (ista & (UNICAM_FSI | UNICAM_LCI) && !(ista & UNICAM_FEI)) {
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + spin_lock(&unicam->node[i].dma_queue_lock);
> + if (!list_empty(&unicam->node[i].dma_queue) &&
> + !unicam->node[i].next_frm)
> + unicam_schedule_next_buffer(&unicam-
> >node[i]);
> + spin_unlock(&unicam->node[i].dma_queue_lock);
> + }
> + }
> +
> + if (reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> + /* Switch out of trigger mode if selected */
> + reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> + reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> + }
> + return IRQ_HANDLED;
> +}
> +
> +static int unicam_querycap(struct file *file, void *priv,
> + struct v4l2_capability *cap)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + strlcpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> + strlcpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
> +
> + snprintf(cap->bus_info, sizeof(cap->bus_info),
> + "platform:%s", dev_name(&dev->pdev->dev));
> +
> + cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE |
> V4L2_CAP_META_CAPTURE;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_fmtdesc *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + unsigned int index = 0;
> + unsigned int i;
> + int ret = 0;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = i,
> + .pad = IMAGE_PAD,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> + const struct unicam_fmt *fmt;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad,
> enum_mbus_code,
> + NULL, &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx %d
> returned %d - index invalid\n",
> + i, ret);
> + return -EINVAL;
> + }
> +
> + fmt = find_format_by_code(mbus_code.code);
> + if (fmt) {
> + if (fmt->fourcc) {
> + if (index == f->index) {
> + f->pixelformat = fmt->fourcc;
> + break;
> + }
> + index++;
> + }
> + if (fmt->repacked_fourcc) {
> + if (index == f->index) {
> + f->pixelformat = fmt-
> >repacked_fourcc;
> + break;
> + }
> + index++;
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int unicam_g_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt = NULL;
> + int ret;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + /*
> + * If a flip has occurred in the sensor, the fmt code might
> have
> + * changed. So we will need to re-fetch the format from the
> subdevice.
> + */
> + ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
> + if (ret)
> + return -EINVAL;
> +
> + /* Find the V4L2 format from mbus code. We must match a known
> format. */
> + fmt = find_format_by_code(mbus_fmt.code);
> + if (!fmt)
> + return -EINVAL;
> +
> + node->fmt = fmt;
> + node->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static
> +const struct unicam_fmt *get_first_supported_format(struct
> unicam_device *dev)
> +{
> + struct v4l2_subdev_mbus_code_enum mbus_code;
> + const struct unicam_fmt *fmt = NULL;
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; ret != -EINVAL && ret != -ENOIOCTLCMD; ++i) {
> + memset(&mbus_code, 0, sizeof(mbus_code));
> + mbus_code.index = i;
> + mbus_code.pad = IMAGE_PAD;
> + mbus_code.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad,
> enum_mbus_code, NULL,
> + &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx %u
> returned %d - continue\n",
> + i, ret);
> + continue;
> + }
> +
> + unicam_dbg(2, dev, "subdev %s: code: 0x%08x idx: %u\n",
> + dev->sensor->name, mbus_code.code, i);
> +
> + fmt = find_format_by_code(mbus_code.code);
> + unicam_dbg(2, dev, "fmt 0x%08x returned as %p, V4L2
> FOURCC 0x%08x, csi_dt 0x%02x\n",
> + mbus_code.code, fmt, fmt ? fmt->fourcc : 0,
> + fmt ? fmt->csi_dt : 0);
> + if (fmt)
> + return fmt;
> + }
> +
> + return NULL;
> +}
> +
> +static int unicam_try_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_TRY,
> + .pad = IMAGE_PAD
> + };
> + struct v4l2_mbus_framefmt *mbus_fmt = &sd_fmt.format;
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
> + if (!fmt) {
> + /*
> + * Pixel format not supported by unicam. Choose the
> first
> + * supported format, and let the sensor choose
> something else.
> + */
> + unicam_dbg(3, dev, "Fourcc format (0x%08x) not found.
> Use first format.\n",
> + f->fmt.pix.pixelformat);
> +
> + fmt = &formats[0];
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + }
> +
> + v4l2_fill_mbus_format(mbus_fmt, &f->fmt.pix, fmt->code);
> + /*
> + * No support for receiving interlaced video, so never
> + * request it from the sensor subdev.
> + */
> + mbus_fmt->field = V4L2_FIELD_NONE;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev-
> >sensor_config,
> + &sd_fmt);
> + if (ret && ret != -ENOIOCTLCMD && ret != -ENODEV)
> + return ret;
> +
> + if (mbus_fmt->field != V4L2_FIELD_NONE)
> + unicam_info(dev, "Sensor trying to send interlaced
> video - results may be unpredictable\n");
> +
> + v4l2_fill_pix_format(&f->fmt.pix, &sd_fmt.format);
> + if (mbus_fmt->code != fmt->code) {
> + /* Sensor has returned an alternate format */
> + fmt = find_format_by_code(mbus_fmt->code);
> + if (!fmt) {
> + /*
> + * The alternate format is one unicam can't
> support.
> + * Find the first format that is supported by
> both, and
> + * then set that.
> + */
> + fmt = get_first_supported_format(dev);
> + mbus_fmt->code = fmt->code;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad,
> set_fmt,
> + dev->sensor_config,
> &sd_fmt);
> + if (ret && ret != -ENOIOCTLCMD && ret !=
> -ENODEV)
> + return ret;
> +
> + if (mbus_fmt->field != V4L2_FIELD_NONE)
> + unicam_info(dev, "Sensor trying to send
> interlaced video - results may be unpredictable\n");
> +
> + v4l2_fill_pix_format(&f->fmt.pix,
> &sd_fmt.format);
> +
> + if (mbus_fmt->code != fmt->code) {
> + /*
> + * We've set a format that the sensor
> reports
> + * as being supported, but it refuses
> to set it.
> + * Not much else we can do.
> + * Assume that the sensor driver may
> accept the
> + * format when it is set (rather than
> tried).
> + */
> + unicam_err(dev, "Sensor won't accept
> default format, and Unicam can't support sensor default\n");
> + }
> + }
> +
> + if (fmt->fourcc)
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + else
> + f->fmt.pix.pixelformat = fmt->repacked_fourcc;
> + }
> +
> + return unicam_calc_format_size_bpl(dev, fmt, f);
> +}
> +
> +static int unicam_s_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct vb2_queue *q = &node->buffer_queue;
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (vb2_is_busy(q))
> + return -EBUSY;
> +
> + ret = unicam_try_fmt_vid_cap(file, priv, f);
> + if (ret < 0)
> + return ret;
> +
> + fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
> + if (!fmt) {
> + /*
> + * Unknown pixel format - adopt a default.
> + * This shouldn't happen as try_fmt should have
> resolved any
> + * issues first.
> + */
> + fmt = get_first_supported_format(dev);
> + if (!fmt)
> + /*
> + * It shouldn't be possible to get here with no
> + * supported formats
> + */
> + return -EINVAL;
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + return -EINVAL;
> + }
> +
> + v4l2_fill_mbus_format(&mbus_fmt, &f->fmt.pix, fmt->code);
> +
> + ret = __subdev_set_format(dev, &mbus_fmt, node->pad_id);
> + if (ret) {
> + unicam_dbg(3, dev, "%s __subdev_set_format failed
> %d\n",
> + __func__, ret);
> + return ret;
> + }
> +
> + /* Just double check nothing has gone wrong */
> + if (mbus_fmt.code != fmt->code) {
> + unicam_dbg(3, dev,
> + "%s subdev changed format on us, this should
> not happen\n",
> + __func__);
> + return -EINVAL;
> + }
> +
> + node->fmt = fmt;
> + node->v_fmt.fmt.pix.pixelformat = f->fmt.pix.pixelformat;
> + node->v_fmt.fmt.pix.bytesperline = f->fmt.pix.bytesperline;
> + unicam_reset_format(node);
> +
> + unicam_dbg(3, dev,
> + "%s %dx%d, mbus_fmt 0x%08X, V4L2 pix 0x%08X.\n",
> + __func__, node->v_fmt.fmt.pix.width,
> + node->v_fmt.fmt.pix.height, mbus_fmt.code,
> + node->v_fmt.fmt.pix.pixelformat);
> +
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_fmt_meta_cap(struct file *file, void *priv,
> + struct v4l2_fmtdesc *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + u32 code;
> + int ret = 0;
> +
> + if (node->pad_id != METADATA_PAD || f->index != 0)
> + return -EINVAL;
> +
> + if (dev->sensor_embedded_data) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = f->index,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = METADATA_PAD,
> + };
> +
> + ret = v4l2_subdev_call(dev->sensor, pad,
> enum_mbus_code, NULL,
> + &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx 0
> returned %d - index invalid\n",
> + ret);
> + return -EINVAL;
> + }
> +
> + code = mbus_code.code;
> + } else {
> + code = MEDIA_BUS_FMT_SENSOR_DATA;
> + }
> +
> + fmt = find_format_by_code(code);
> + if (fmt)
> + f->pixelformat = fmt->fourcc;
> +
> + return 0;
> +}
> +
> +static int unicam_g_fmt_meta_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> +
> + if (node->pad_id != METADATA_PAD)
> + return -EINVAL;
> +
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static int unicam_queue_setup(struct vb2_queue *vq,
> + unsigned int *nbuffers,
> + unsigned int *nplanes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> + unsigned int size = node->pad_id == IMAGE_PAD ?
> + node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> +
> + if (vq->num_buffers + *nbuffers < 3)
> + *nbuffers = 3 - vq->num_buffers;
> +
> + if (*nplanes) {
> + if (sizes[0] < size) {
> + unicam_err(dev, "sizes[0] %i < size %u\n",
> sizes[0],
> + size);
> + return -EINVAL;
> + }
> + size = sizes[0];
> + }
> +
> + *nplanes = 1;
> + sizes[0] = size;
> +
> + return 0;
> +}
> +
> +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct unicam_device *dev = node->dev;
> + struct unicam_buffer *buf = to_unicam_buffer(vb);
> + unsigned long size;
> +
> + if (WARN_ON(!node->fmt))
> + return -EINVAL;
> +
> + size = node->pad_id == IMAGE_PAD ? node-
> >v_fmt.fmt.pix.sizeimage :
> + node-
> >v_fmt.fmt.meta.buffersize;
> + if (vb2_plane_size(vb, 0) < size) {
> + unicam_err(dev, "data will not fit into plane (%lu <
> %lu)\n",
> + vb2_plane_size(vb, 0), size);
> + return -EINVAL;
> + }
> +
> + vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> + return 0;
> +}
> +
> +static void unicam_buffer_queue(struct vb2_buffer *vb)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct unicam_buffer *buf = to_unicam_buffer(vb);
> + unsigned long flags;
> +
> + spin_lock_irqsave(&node->dma_queue_lock, flags);
> + list_add_tail(&buf->list, &node->dma_queue);
> + spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +}
> +
> +static void unicam_set_packing_config(struct unicam_device *dev)
> +{
> + u32 pack, unpack;
> + u32 val;
> +
> + if (dev->node[IMAGE_PAD].v_fmt.fmt.pix.pixelformat ==
> + dev->node[IMAGE_PAD].fmt->fourcc) {
> + unpack = UNICAM_PUM_NONE;
> + pack = UNICAM_PPM_NONE;
> + } else {
> + switch (dev->node[IMAGE_PAD].fmt->depth) {
> + case 8:
> + unpack = UNICAM_PUM_UNPACK8;
> + break;
> + case 10:
> + unpack = UNICAM_PUM_UNPACK10;
> + break;
> + case 12:
> + unpack = UNICAM_PUM_UNPACK12;
> + break;
> + case 14:
> + unpack = UNICAM_PUM_UNPACK14;
> + break;
> + case 16:
> + unpack = UNICAM_PUM_UNPACK16;
> + break;
> + default:
> + unpack = UNICAM_PUM_NONE;
> + break;
> + }
> +
> + /* Repacking is always to 16bpp */
> + pack = UNICAM_PPM_PACK16;
> + }
> +
> + val = 0;
> + set_field(&val, unpack, UNICAM_PUM_MASK);
> + set_field(&val, pack, UNICAM_PPM_MASK);
> + reg_write(dev, UNICAM_IPIPE, val);
> +}
> +
> +static void unicam_cfg_image_id(struct unicam_device *dev)
> +{
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 mode, hardcode VC 0 for now. */
> + reg_write(dev, UNICAM_IDI0,
> + (0 << 6) | dev->node[IMAGE_PAD].fmt->csi_dt);
> + } else {
> + /* CCP2 mode */
> + reg_write(dev, UNICAM_IDI0,
> + 0x80 | dev->node[IMAGE_PAD].fmt->csi_dt);
> + }
> +}
> +
> +static void unicam_enable_ed(struct unicam_device *dev)
> +{
> + u32 val = reg_read(dev, UNICAM_DCS);
> +
> + set_field(&val, 2, UNICAM_EDL_MASK);
> + /* Do not wrap at the end of the embedded data buffer */
> + set_field(&val, 0, UNICAM_DBOB);
> +
> + reg_write(dev, UNICAM_DCS, val);
> +}
> +
> +static void unicam_start_rx(struct unicam_device *dev, dma_addr_t
> *addr)
> +{
> + int line_int_freq = dev->node[IMAGE_PAD].v_fmt.fmt.pix.height
> >> 2;
> + unsigned int size, i;
> + u32 val;
> +
> + if (line_int_freq < 128)
> + line_int_freq = 128;
> +
> + /* Enable lane clocks */
> + val = 1;
> + for (i = 0; i < dev->active_data_lanes; i++)
> + val = val << 2 | 1;
> + clk_write(dev, val);
> +
> + /* Basic init */
> + reg_write(dev, UNICAM_CTRL, UNICAM_MEM);
> +
> + /* Enable analogue control, and leave in reset. */
> + val = UNICAM_AR;
> + set_field(&val, 7, UNICAM_CTATADJ_MASK);
> + set_field(&val, 7, UNICAM_PTATADJ_MASK);
> + reg_write(dev, UNICAM_ANA, val);
> + usleep_range(1000, 2000);
> +
> + /* Come out of reset */
> + reg_write_field(dev, UNICAM_ANA, 0, UNICAM_AR);
> +
> + /* Peripheral reset */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> + /* Enable Rx control. */
> + val = reg_read(dev, UNICAM_CTRL);
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> + set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> + } else {
> + set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> + set_field(&val, dev->bus_flags, UNICAM_DCM_MASK);
> + }
> + /* Packet framer timeout */
> + set_field(&val, 0xf, UNICAM_PFT_MASK);
> + set_field(&val, 128, UNICAM_OET_MASK);
> + reg_write(dev, UNICAM_CTRL, val);
> +
> + reg_write(dev, UNICAM_IHWIN, 0);
> + reg_write(dev, UNICAM_IVWIN, 0);
> +
> + /* AXI bus access QoS setup */
> + val = reg_read(dev, UNICAM_PRI);
> + set_field(&val, 0, UNICAM_BL_MASK);
> + set_field(&val, 0, UNICAM_BS_MASK);
> + set_field(&val, 0xe, UNICAM_PP_MASK);
> + set_field(&val, 8, UNICAM_NP_MASK);
> + set_field(&val, 2, UNICAM_PT_MASK);
> + set_field(&val, 1, UNICAM_PE);
> + reg_write(dev, UNICAM_PRI, val);
> +
> + reg_write_field(dev, UNICAM_ANA, 0, UNICAM_DDL);
> +
> + /* Always start in trigger frame capture mode (UNICAM_FCM set)
> */
> + val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> + set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> + reg_write(dev, UNICAM_ICTL, val);
> + reg_write(dev, UNICAM_STA, UNICAM_STA_MASK_ALL);
> + reg_write(dev, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> +
> + /* tclk_term_en */
> + reg_write_field(dev, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> + /* tclk_settle */
> + reg_write_field(dev, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> + /* td_term_en */
> + reg_write_field(dev, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> + /* ths_settle */
> + reg_write_field(dev, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> + /* trx_enable */
> + reg_write_field(dev, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> +
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_SOE);
> +
> + /* Packet compare setup - required to avoid missing frame ends
> */
> + val = 0;
> + set_field(&val, 1, UNICAM_PCE);
> + set_field(&val, 1, UNICAM_GI);
> + set_field(&val, 1, UNICAM_CPH);
> + set_field(&val, 0, UNICAM_PCVC_MASK);
> + set_field(&val, 1, UNICAM_PCDT_MASK);
> + reg_write(dev, UNICAM_CMP0, val);
> +
> + /* Enable clock lane and set up terminations */
> + val = 0;
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 */
> + set_field(&val, 1, UNICAM_CLE);
> + set_field(&val, 1, UNICAM_CLLPE);
> + if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
> + set_field(&val, 1, UNICAM_CLTRE);
> + set_field(&val, 1, UNICAM_CLHSE);
> + }
> + } else {
> + /* CCP2 */
> + set_field(&val, 1, UNICAM_CLE);
> + set_field(&val, 1, UNICAM_CLHSE);
> + set_field(&val, 1, UNICAM_CLTRE);
> + }
> + reg_write(dev, UNICAM_CLK, val);
> +
> + /*
> + * Enable required data lanes with appropriate terminations.
> + * The same value needs to be written to UNICAM_DATn registers
> for
> + * the active lanes, and 0 for inactive ones.
> + */
> + val = 0;
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 */
> + set_field(&val, 1, UNICAM_DLE);
> + set_field(&val, 1, UNICAM_DLLPE);
> + if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
> + set_field(&val, 1, UNICAM_DLTRE);
> + set_field(&val, 1, UNICAM_DLHSE);
> + }
> + } else {
> + /* CCP2 */
> + set_field(&val, 1, UNICAM_DLE);
> + set_field(&val, 1, UNICAM_DLHSE);
> + set_field(&val, 1, UNICAM_DLTRE);
> + }
> + reg_write(dev, UNICAM_DAT0, val);
> +
> + if (dev->active_data_lanes == 1)
> + val = 0;
> + reg_write(dev, UNICAM_DAT1, val);
> +
> + if (dev->max_data_lanes > 2) {
> + /*
> + * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if
> the
> + * instance supports more than 2 data lanes.
> + */
> + if (dev->active_data_lanes == 2)
> + val = 0;
> + reg_write(dev, UNICAM_DAT2, val);
> +
> + if (dev->active_data_lanes == 3)
> + val = 0;
> + reg_write(dev, UNICAM_DAT3, val);
> + }
> +
> + reg_write(dev, UNICAM_IBLS,
> + dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline);
> + size = dev->node[IMAGE_PAD].v_fmt.fmt.pix.sizeimage;
> + unicam_wr_dma_addr(dev, addr[IMAGE_PAD], size, IMAGE_PAD);
> + unicam_set_packing_config(dev);
> + unicam_cfg_image_id(dev);
> +
> + val = reg_read(dev, UNICAM_MISC);
> + set_field(&val, 1, UNICAM_FL0);
> + set_field(&val, 1, UNICAM_FL1);
> + reg_write(dev, UNICAM_MISC, val);
> +
> + if (dev->node[METADATA_PAD].streaming && dev-
> >sensor_embedded_data) {
> + size = dev-
> >node[METADATA_PAD].v_fmt.fmt.meta.buffersize;
> + unicam_enable_ed(dev);
> + unicam_wr_dma_addr(dev, addr[METADATA_PAD], size,
> METADATA_PAD);
> + }
> +
> + /* Enable peripheral */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPE);
> +
> + /* Load image pointers */
> + reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> +
> + /* Load embedded data buffer pointers if needed */
> + if (dev->node[METADATA_PAD].streaming && dev-
> >sensor_embedded_data)
> + reg_write_field(dev, UNICAM_DCS, 1, UNICAM_LDP);
> +
> + /*
> + * Enable trigger only for the first frame to
> + * sync correctly to the FS from the source.
> + */
> + reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_TFC);
> +}
> +
> +static void unicam_disable(struct unicam_device *dev)
> +{
> + /* Analogue lane control disable */
> + reg_write_field(dev, UNICAM_ANA, 1, UNICAM_DDL);
> +
> + /* Stop the output engine */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_SOE);
> +
> + /* Disable the data lanes. */
> + reg_write(dev, UNICAM_DAT0, 0);
> + reg_write(dev, UNICAM_DAT1, 0);
> +
> + if (dev->max_data_lanes > 2) {
> + reg_write(dev, UNICAM_DAT2, 0);
> + reg_write(dev, UNICAM_DAT3, 0);
> + }
> +
> + /* Peripheral reset */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
> + usleep_range(50, 100);
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> + /* Disable peripheral */
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> + /* Clear ED setup */
> + reg_write(dev, UNICAM_DCS, 0);
> +
> + /* Disable all lane clocks */
> + clk_write(dev, 0);
> +}
> +
> +static void unicam_return_buffers(struct unicam_node *node)
> +{
> + struct unicam_buffer *buf, *tmp;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&node->dma_queue_lock, flags);
> + list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> + list_del(&buf->list);
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> + }
> +
> + if (node->cur_frm)
> + vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> + VB2_BUF_STATE_ERROR);
> + if (node->next_frm && node->cur_frm != node->next_frm)
> + vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> + VB2_BUF_STATE_ERROR);
> +
> + node->cur_frm = NULL;
> + node->next_frm = NULL;
> + spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +}
> +
> +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int
> count)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> + dma_addr_t buffer_addr[MAX_NODES] = { 0 };
> + unsigned long flags;
> + unsigned int i;
> + int ret;
> +
> + node->streaming = true;
> + if (!unicam_all_nodes_streaming(dev)) {
> + unicam_dbg(3, dev, "Not all nodes are streaming yet.");
> + return 0;
> + }
> +
> + dev->sequence = 0;
> + ret = unicam_runtime_get(dev);
> + if (ret < 0) {
> + unicam_dbg(3, dev, "unicam_runtime_get failed\n");
> + goto err_streaming;
> + }
> +
> + /*
> + * TODO: Retrieve the number of active data lanes from the
> connected
> + * subdevice.
> + */
> + dev->active_data_lanes = dev->max_data_lanes;
> +
> + ret = clk_set_rate(dev->clock, 100 * 1000 * 1000);
> + if (ret) {
> + unicam_err(dev, "failed to set up clock\n");
> + goto err_pm_put;
> + }
> +
> + ret = clk_prepare_enable(dev->clock);
> + if (ret) {
> + unicam_err(dev, "Failed to enable CSI clock: %d\n",
> ret);
> + goto err_pm_put;
> + }
> +
> + for (i = 0; i < ARRAY_SIZE(dev->node); i++) {
> + struct unicam_buffer *buf;
> +
> + if (!dev->node[i].streaming)
> + continue;
> +
> + spin_lock_irqsave(&dev->node[i].dma_queue_lock, flags);
> + buf = list_first_entry(&dev->node[i].dma_queue,
> + struct unicam_buffer, list);
> + dev->node[i].cur_frm = buf;
> + dev->node[i].next_frm = buf;
> + list_del(&buf->list);
> + spin_unlock_irqrestore(&dev->node[i].dma_queue_lock,
> flags);
> +
> + buffer_addr[i] =
> + vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf,
> 0);
> + }
> +
> + unicam_start_rx(dev, buffer_addr);
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_stream, 1);
> + if (ret < 0) {
> + unicam_err(dev, "stream on failed in subdev\n");
> + goto err_disable_unicam;
> + }
> +
> + return 0;
> +
> +err_disable_unicam:
> + unicam_disable(dev);
> + clk_disable_unprepare(dev->clock);
> +err_pm_put:
> + unicam_runtime_put(dev);
> +err_streaming:
> + unicam_return_buffers(node);
> + node->streaming = false;
> +
> + return ret;
> +}
> +
> +static void unicam_stop_streaming(struct vb2_queue *vq)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> +
> + node->streaming = false;
> +
> + if (node->pad_id == IMAGE_PAD) {
> + /*
> + * Stop streaming the sensor and disable the
> peripheral.
> + * We cannot continue streaming embedded data with the
> + * image pad disabled.
> + */
> + if (v4l2_subdev_call(dev->sensor, video, s_stream, 0) <
> 0)
> + unicam_err(dev, "stream off failed in
> subdev\n");
> +
> + unicam_disable(dev);
> + clk_disable_unprepare(dev->clock);
> + unicam_runtime_put(dev);
> +
> + } else if (node->pad_id == METADATA_PAD) {
> + /*
> + * Allow the hardware to spin in the dummy buffer.
> + * This is only really needed if the embedded data pad
> is
> + * disabled before the image pad.
> + */
> + unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr,
> + DUMMY_BUF_SIZE, METADATA_PAD);
> + }
> +
> + /* Clear all queued buffers for the node */
> + unicam_return_buffers(node);
> +}
> +
> +static int unicam_enum_input(struct file *file, void *priv,
> + struct v4l2_input *inp)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + if (inp->index != 0)
> + return -EINVAL;
> +
> + inp->type = V4L2_INPUT_TYPE_CAMERA;
> + if (v4l2_subdev_has_op(dev->sensor, video, s_dv_timings)) {
> + inp->capabilities = V4L2_IN_CAP_DV_TIMINGS;
> + inp->std = 0;
> + } else if (v4l2_subdev_has_op(dev->sensor, video, s_std)) {
> + inp->capabilities = V4L2_IN_CAP_STD;
> + if (v4l2_subdev_call(dev->sensor, video, g_tvnorms,
> &inp->std)
> + < 0)
> + inp->std = V4L2_STD_ALL;
> + } else {
> + inp->capabilities = 0;
> + inp->std = 0;
> + }
> + sprintf(inp->name, "Camera 0");
> + return 0;
> +}
> +
> +static int unicam_g_input(struct file *file, void *priv, unsigned
> int *i)
> +{
> + *i = 0;
> +
> + return 0;
> +}
> +
> +static int unicam_s_input(struct file *file, void *priv, unsigned
> int i)
> +{
> + /*
> + * FIXME: Ideally we would like to be able to query the source
> + * subdevice for information over the input connectors it
> supports,
> + * and map that through in to a call to video_ops->s_routing.
> + * There is no infrastructure support for defining that within
> + * devicetree at present. Until that is implemented we can't
> + * map a user physical connector number to s_routing input
> number.
> + */
> + if (i > 0)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int unicam_querystd(struct file *file, void *priv,
> + v4l2_std_id *std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, querystd, std);
> +}
> +
> +static int unicam_g_std(struct file *file, void *priv, v4l2_std_id
> *std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, g_std, std);
> +}
> +
> +static int unicam_s_std(struct file *file, void *priv, v4l2_std_id
> std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + int ret;
> + v4l2_std_id current_std;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, g_std,
> ¤t_std);
> + if (ret)
> + return ret;
> +
> + if (std == current_std)
> + return 0;
> +
> + if (vb2_is_busy(&node->buffer_queue))
> + return -EBUSY;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_std, std);
> +
> + /* Force recomputation of bytesperline */
> + node->v_fmt.fmt.pix.bytesperline = 0;
> +
> + unicam_reset_format(node);
> +
> + return ret;
> +}
> +
> +static int unicam_s_edid(struct file *file, void *priv, struct
> v4l2_edid *edid)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, set_edid, edid);
> +}
> +
> +static int unicam_g_edid(struct file *file, void *priv, struct
> v4l2_edid *edid)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, get_edid, edid);
> +}
> +
> +static int unicam_s_selection(struct file *file, void *priv,
> + struct v4l2_selection *sel)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_selection sdsel = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .target = sel->target,
> + .flags = sel->flags,
> + .r = sel->r,
> + };
> +
> + return v4l2_subdev_call(dev->sensor, pad, set_selection, NULL,
> &sdsel);
> +}
> +
> +static int unicam_g_selection(struct file *file, void *priv,
> + struct v4l2_selection *sel)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_selection sdsel = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .target = sel->target,
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, get_selection, NULL,
> &sdsel);
> + if (!ret)
> + sel->r = sdsel.r;
> +
> + return ret;
> +}
> +
> +static int unicam_enum_framesizes(struct file *file, void *priv,
> + struct v4l2_frmsizeenum *fsize)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + struct v4l2_subdev_frame_size_enum fse;
> + int ret;
> +
> + /* check for valid format */
> + fmt = find_format_by_pix(dev, fsize->pixel_format);
> + if (!fmt) {
> + unicam_dbg(3, dev, "Invalid pixel code: %x\n",
> + fsize->pixel_format);
> + return -EINVAL;
> + }
> + fse.code = fmt->code;
> +
> + fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> + fse.index = fsize->index;
> + fse.pad = node->pad_id;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_size, NULL,
> &fse);
> + if (ret)
> + return ret;
> +
> + unicam_dbg(1, dev, "%s: index: %d code: %x W:[%d,%d]
> H:[%d,%d]\n",
> + __func__, fse.index, fse.code, fse.min_width,
> fse.max_width,
> + fse.min_height, fse.max_height);
> +
> + fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
> + fsize->discrete.width = fse.max_width;
> + fsize->discrete.height = fse.max_height;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_frameintervals(struct file *file, void *priv,
> + struct v4l2_frmivalenum *fival)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + struct v4l2_subdev_frame_interval_enum fie = {
> + .index = fival->index,
> + .width = fival->width,
> + .height = fival->height,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> + int ret;
> +
> + fmt = find_format_by_pix(dev, fival->pixel_format);
> + if (!fmt)
> + return -EINVAL;
> +
> + fie.code = fmt->code;
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_interval,
> + NULL, &fie);
> + if (ret)
> + return ret;
> +
> + fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
> + fival->discrete = fie.interval;
> +
> + return 0;
> +}
> +
> +static int unicam_g_parm(struct file *file, void *fh, struct
> v4l2_streamparm *a)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_g_parm_cap(video_devdata(file), dev->sensor, a);
> +}
> +
> +static int unicam_s_parm(struct file *file, void *fh, struct
> v4l2_streamparm *a)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_s_parm_cap(video_devdata(file), dev->sensor, a);
> +}
> +
> +static int unicam_g_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, g_dv_timings,
> timings);
> +}
> +
> +static int unicam_s_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_dv_timings current_timings;
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, g_dv_timings,
> + ¤t_timings);
> +
> + if (v4l2_match_dv_timings(timings, ¤t_timings, 0, false))
> + return 0;
> +
> + if (vb2_is_busy(&node->buffer_queue))
> + return -EBUSY;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_dv_timings,
> timings);
> +
> + /* Force recomputation of bytesperline */
> + node->v_fmt.fmt.pix.bytesperline = 0;
> +
> + unicam_reset_format(node);
> +
> + return ret;
> +}
> +
> +static int unicam_query_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, query_dv_timings,
> timings);
> +}
> +
> +static int unicam_enum_dv_timings(struct file *file, void *priv,
> + struct v4l2_enum_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, enum_dv_timings,
> timings);
> +}
> +
> +static int unicam_dv_timings_cap(struct file *file, void *priv,
> + struct v4l2_dv_timings_cap *cap)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, dv_timings_cap, cap);
> +}
> +
> +static int unicam_subscribe_event(struct v4l2_fh *fh,
> + const struct v4l2_event_subscription
> *sub)
> +{
> + switch (sub->type) {
> + case V4L2_EVENT_FRAME_SYNC:
> + return v4l2_event_subscribe(fh, sub, 2, NULL);
> + case V4L2_EVENT_SOURCE_CHANGE:
> + return v4l2_event_subscribe(fh, sub, 4, NULL);
> + }
> +
> + return v4l2_ctrl_subscribe_event(fh, sub);
> +}
> +
> +static int unicam_log_status(struct file *file, void *fh)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + u32 reg;
> +
> + /* status for sub devices */
> + v4l2_device_call_all(&dev->v4l2_dev, 0, core, log_status);
> +
> + unicam_info(dev, "-----Receiver status-----\n");
> + unicam_info(dev, "V4L2 width/height: %ux%u\n",
> + node->v_fmt.fmt.pix.width, node-
> >v_fmt.fmt.pix.height);
> + unicam_info(dev, "Mediabus format: %08x\n", node->fmt-
> >code);
> + unicam_info(dev, "V4L2 format: %08x\n",
> + node->v_fmt.fmt.pix.pixelformat);
> + reg = reg_read(dev, UNICAM_IPIPE);
> + unicam_info(dev, "Unpacking/packing: %u / %u\n",
> + get_field(reg, UNICAM_PUM_MASK),
> + get_field(reg, UNICAM_PPM_MASK));
> + unicam_info(dev, "----Live data----\n");
> + unicam_info(dev, "Programmed stride: %4u\n",
> + reg_read(dev, UNICAM_IBLS));
> + unicam_info(dev, "Detected resolution: %ux%u\n",
> + reg_read(dev, UNICAM_IHSTA),
> + reg_read(dev, UNICAM_IVSTA));
> + unicam_info(dev, "Write pointer: %08x\n",
> + reg_read(dev, UNICAM_IBWP));
> +
> + return 0;
> +}
> +
> +static void unicam_notify(struct v4l2_subdev *sd,
> + unsigned int notification, void *arg)
> +{
> + struct unicam_device *dev = to_unicam_device(sd->v4l2_dev);
> +
> + switch (notification) {
> + case V4L2_DEVICE_NOTIFY_EVENT:
> + v4l2_event_queue(&dev->node[IMAGE_PAD].video_dev, arg);
> + break;
> + default:
> + break;
> + }
> +}
> +
> +static const struct vb2_ops unicam_video_qops = {
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .queue_setup = unicam_queue_setup,
> + .buf_prepare = unicam_buffer_prepare,
> + .buf_queue = unicam_buffer_queue,
> + .start_streaming = unicam_start_streaming,
> + .stop_streaming = unicam_stop_streaming,
> +};
> +
> +/*
> + * unicam_v4l2_open : This function is based on the v4l2_fh_open
> helper
> + * function. It has been augmented to handle sensor subdevice power
> management,
> + */
> +static int unicam_v4l2_open(struct file *file)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + int ret;
> +
> + mutex_lock(&node->lock);
> +
> + ret = v4l2_fh_open(file);
> + if (ret) {
> + unicam_err(dev, "v4l2_fh_open failed\n");
> + goto unlock;
> + }
> +
> + node->open++;
> +
> + if (!v4l2_fh_is_singular_file(file))
> + goto unlock;
> +
> + ret = v4l2_subdev_call(dev->sensor, core, s_power, 1);
> + if (ret < 0 && ret != -ENOIOCTLCMD) {
> + v4l2_fh_release(file);
> + node->open--;
> + goto unlock;
> + }
> +
> + ret = 0;
> +
> +unlock:
> + mutex_unlock(&node->lock);
> + return ret;
> +}
> +
> +static int unicam_v4l2_release(struct file *file)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev *sd = dev->sensor;
> + bool fh_singular;
> + int ret;
> +
> + mutex_lock(&node->lock);
> +
> + fh_singular = v4l2_fh_is_singular_file(file);
> +
> + ret = _vb2_fop_release(file, NULL);
> +
> + if (fh_singular)
> + v4l2_subdev_call(sd, core, s_power, 0);
> +
> + node->open--;
> + mutex_unlock(&node->lock);
> +
> + return ret;
> +}
> +
> +/* unicam capture driver file operations */
> +static const struct v4l2_file_operations unicam_fops = {
> + .owner = THIS_MODULE,
> + .open = unicam_v4l2_open,
> + .release = unicam_v4l2_release,
> + .read = vb2_fop_read,
> + .poll = vb2_fop_poll,
> + .unlocked_ioctl = video_ioctl2,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +/* unicam capture ioctl operations */
> +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> + .vidioc_querycap = unicam_querycap,
> + .vidioc_enum_fmt_vid_cap = unicam_enum_fmt_vid_cap,
> + .vidioc_g_fmt_vid_cap = unicam_g_fmt_vid_cap,
> + .vidioc_s_fmt_vid_cap = unicam_s_fmt_vid_cap,
> + .vidioc_try_fmt_vid_cap = unicam_try_fmt_vid_cap,
> +
> + .vidioc_enum_fmt_meta_cap = unicam_enum_fmt_meta_cap,
> + .vidioc_g_fmt_meta_cap = unicam_g_fmt_meta_cap,
> + .vidioc_s_fmt_meta_cap = unicam_g_fmt_meta_cap,
> + .vidioc_try_fmt_meta_cap = unicam_g_fmt_meta_cap,
> +
> + .vidioc_enum_input = unicam_enum_input,
> + .vidioc_g_input = unicam_g_input,
> + .vidioc_s_input = unicam_s_input,
> +
> + .vidioc_querystd = unicam_querystd,
> + .vidioc_s_std = unicam_s_std,
> + .vidioc_g_std = unicam_g_std,
> +
> + .vidioc_g_edid = unicam_g_edid,
> + .vidioc_s_edid = unicam_s_edid,
> +
> + .vidioc_enum_framesizes = unicam_enum_framesizes,
> + .vidioc_enum_frameintervals = unicam_enum_frameintervals,
> +
> + .vidioc_g_selection = unicam_g_selection,
> + .vidioc_s_selection = unicam_s_selection,
> +
> + .vidioc_g_parm = unicam_g_parm,
> + .vidioc_s_parm = unicam_s_parm,
> +
> + .vidioc_s_dv_timings = unicam_s_dv_timings,
> + .vidioc_g_dv_timings = unicam_g_dv_timings,
> + .vidioc_query_dv_timings = unicam_query_dv_timings,
> + .vidioc_enum_dv_timings = unicam_enum_dv_timings,
> + .vidioc_dv_timings_cap = unicam_dv_timings_cap,
> +
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +
> + .vidioc_log_status = unicam_log_status,
> + .vidioc_subscribe_event = unicam_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static int
> +unicam_async_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_subdev *asd)
> +{
> + struct unicam_device *unicam = to_unicam_device(notifier-
> >v4l2_dev);
> +
> + if (unicam->sensor) {
> + unicam_info(unicam, "Rejecting subdev %s (Already
> set!!)",
> + subdev->name);
> + return 0;
> + }
> +
> + unicam->sensor = subdev;
> + unicam_dbg(1, unicam, "Using sensor %s for capture\n", subdev-
> >name);
> +
> + return 0;
> +}
> +
> +static void unicam_release(struct kref *kref)
> +{
> + struct unicam_device *unicam =
> + container_of(kref, struct unicam_device, kref);
> +
> + v4l2_ctrl_handler_free(&unicam->ctrl_handler);
> + media_device_cleanup(&unicam->mdev);
> +
> + if (unicam->sensor_config)
> + v4l2_subdev_free_pad_config(unicam->sensor_config);
> +
> + kfree(unicam);
> +}
> +
> +static void unicam_put(struct unicam_device *unicam)
> +{
> + kref_put(&unicam->kref, unicam_release);
> +}
> +
> +static void unicam_get(struct unicam_device *unicam)
> +{
> + kref_get(&unicam->kref);
> +}
> +
> +static void unicam_node_release(struct video_device *vdev)
> +{
> + struct unicam_node *node = video_get_drvdata(vdev);
> +
> + unicam_put(node->dev);
> +}
> +
> +static int register_node(struct unicam_device *unicam, struct
> unicam_node *node,
> + enum v4l2_buf_type type, int pad_id)
> +{
> + struct video_device *vdev;
> + struct vb2_queue *q;
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (pad_id == IMAGE_PAD) {
> + ret = __subdev_get_format(unicam, &mbus_fmt, pad_id);
> + if (ret) {
> + unicam_err(unicam, "Failed to get_format - ret
> %d\n",
> + ret);
> + return ret;
> + }
> +
> + fmt = find_format_by_code(mbus_fmt.code);
> + if (!fmt) {
> + /*
> + * Find the first format that the sensor and
> unicam both
> + * support
> + */
> + fmt = get_first_supported_format(unicam);
> +
> + if (!fmt)
> + /* No compatible formats */
> + return -EINVAL;
> +
> + mbus_fmt.code = fmt->code;
> + ret = __subdev_set_format(unicam, &mbus_fmt,
> pad_id);
> + if (ret)
> + return -EINVAL;
> + }
> + if (mbus_fmt.field != V4L2_FIELD_NONE) {
> + /* Interlaced not supported - disable it now.
> */
> + mbus_fmt.field = V4L2_FIELD_NONE;
> + ret = __subdev_set_format(unicam, &mbus_fmt,
> pad_id);
> + if (ret)
> + return -EINVAL;
> + }
> +
> + node->v_fmt.fmt.pix.pixelformat = fmt->fourcc ? fmt-
> >fourcc
> + : fmt->repacked_fourcc;
> + } else {
> + /* Fix this node format as embedded data. */
> + fmt = find_format_by_code(MEDIA_BUS_FMT_SENSOR_DATA);
> + node->v_fmt.fmt.meta.dataformat = fmt->fourcc;
> + }
> +
> + node->dev = unicam;
> + node->pad_id = pad_id;
> + node->fmt = fmt;
> +
> + /* Read current subdev format */
> + unicam_reset_format(node);
> +
> + if (v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
> + v4l2_std_id tvnorms;
> +
> + if (WARN_ON(!v4l2_subdev_has_op(unicam->sensor, video,
> + g_tvnorms)))
> + /*
> + * Subdevice should not advertise s_std but not
> + * g_tvnorms
> + */
> + return -EINVAL;
> +
> + ret = v4l2_subdev_call(unicam->sensor, video,
> + g_tvnorms, &tvnorms);
> + if (WARN_ON(ret))
> + return -EINVAL;
> + node->video_dev.tvnorms |= tvnorms;
> + }
> +
> + spin_lock_init(&node->dma_queue_lock);
> + mutex_init(&node->lock);
> +
> + vdev = &node->video_dev;
> + if (pad_id == IMAGE_PAD) {
> + /* Add controls from the subdevice */
> + ret = v4l2_ctrl_add_handler(&unicam->ctrl_handler,
> + unicam->sensor-
> >ctrl_handler, NULL,
> + true);
> + if (ret < 0)
> + return ret;
> +
> + /*
> + * If the sensor subdevice has any controls, associate
> the node
> + * with the ctrl handler to allow access from
> userland.
> + */
> + if (!list_empty(&unicam->ctrl_handler.ctrls))
> + vdev->ctrl_handler = &unicam->ctrl_handler;
> + }
> +
> + q = &node->buffer_queue;
> + q->type = type;
> + q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;
> + q->drv_priv = node;
> + q->ops = &unicam_video_qops;
> + q->mem_ops = &vb2_dma_contig_memops;
> + q->buf_struct_size = sizeof(struct unicam_buffer);
> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + q->lock = &node->lock;
> + q->min_buffers_needed = 2;
> + q->dev = &unicam->pdev->dev;
> +
> + ret = vb2_queue_init(q);
> + if (ret) {
> + unicam_err(unicam, "vb2_queue_init() failed\n");
> + return ret;
> + }
> +
> + INIT_LIST_HEAD(&node->dma_queue);
> +
> + vdev->release = unicam_node_release;
> + vdev->fops = &unicam_fops;
> + vdev->ioctl_ops = &unicam_ioctl_ops;
> + vdev->v4l2_dev = &unicam->v4l2_dev;
> + vdev->vfl_dir = VFL_DIR_RX;
> + vdev->queue = q;
> + vdev->lock = &node->lock;
> + vdev->device_caps = (pad_id == IMAGE_PAD) ?
> + (V4L2_CAP_VIDEO_CAPTURE |
> V4L2_CAP_STREAMING) :
> + (V4L2_CAP_META_CAPTURE |
> V4L2_CAP_STREAMING);
> +
> + /* Define the device names */
> + snprintf(vdev->name, sizeof(vdev->name), "%s-%s",
> UNICAM_MODULE_NAME,
> + pad_id == IMAGE_PAD ? "image" : "embedded");
> +
> + video_set_drvdata(vdev, node);
> + if (pad_id == IMAGE_PAD)
> + vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> + node->pad.flags = MEDIA_PAD_FL_SINK;
> + media_entity_pads_init(&vdev->entity, 1, &node->pad);
> +
> + node->dummy_buf_cpu_addr = dma_alloc_coherent(&unicam->pdev-
> >dev,
> + DUMMY_BUF_SIZE,
> + &node-
> >dummy_buf_dma_addr,
> + GFP_KERNEL);
> + if (!node->dummy_buf_cpu_addr) {
> + unicam_err(unicam, "Unable to allocate dummy
> buffer.\n");
> + return -ENOMEM;
> + }
> +
> + if (pad_id == METADATA_PAD) {
> + v4l2_disable_ioctl(vdev, VIDIOC_DQEVENT);
> + v4l2_disable_ioctl(vdev, VIDIOC_SUBSCRIBE_EVENT);
> + v4l2_disable_ioctl(vdev, VIDIOC_UNSUBSCRIBE_EVENT);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_STD);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_STD);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUMSTD);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, querystd))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_QUERYSTD);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_dv_timings)) {
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_EDID);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_EDID);
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_DV_TIMINGS_CAP);
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_G_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_S_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_ENUM_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_QUERY_DV_TIMINGS);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad,
> enum_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev,
> + VIDIOC_ENUM_FRAMEINTERVALS);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video,
> g_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_PARM);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video,
> s_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_PARM);
> +
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, enum_frame_size))
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_ENUM_FRAMESIZES);
> +
> + if (node->pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, set_selection))
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_S_SELECTION);
> +
> + if (node->pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, get_selection))
> + v4l2_disable_ioctl(&node->video_dev,
> VIDIOC_G_SELECTION);
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + unicam_err(unicam, "Unable to register video device
> %s\n",
> + vdev->name);
> + return ret;
> + }
> +
> + /*
> + * Acquire a reference to unicam, which will be released when
> the video
> + * device will be unregistered and userspace will have closed
> all open
> + * file handles.
> + */
> + unicam_get(unicam);
> + node->registered = true;
> +
> + if (pad_id != METADATA_PAD || unicam->sensor_embedded_data) {
> + ret = media_create_pad_link(&unicam->sensor->entity,
> pad_id,
> + &node->video_dev.entity, 0,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
> + if (ret)
> + unicam_err(unicam, "Unable to create pad link
> for %s\n",
> + vdev->name);
> + }
> +
> + return ret;
> +}
> +
> +static void unregister_nodes(struct unicam_device *unicam)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + struct unicam_node *node = &unicam->node[i];
> +
> + if (node->dummy_buf_cpu_addr) {
> + dma_free_coherent(&unicam->pdev->dev,
> DUMMY_BUF_SIZE,
> + node->dummy_buf_cpu_addr,
> + node->dummy_buf_dma_addr);
> + }
> +
> + if (node->registered) {
> + node->registered = false;
> + video_unregister_device(&node->video_dev);
> + }
> + }
> +}
> +
> +static int unicam_probe_complete(struct unicam_device *unicam)
> +{
> + int ret;
> +
> + unicam->v4l2_dev.notify = unicam_notify;
> +
> + unicam->sensor_config = v4l2_subdev_alloc_pad_config(unicam-
> >sensor);
> + if (!unicam->sensor_config)
> + return -ENOMEM;
> +
> + unicam->sensor_embedded_data = (unicam->sensor->entity.num_pads
> >= 2);
> +
> + ret = register_node(unicam, &unicam->node[IMAGE_PAD],
> + V4L2_BUF_TYPE_VIDEO_CAPTURE, IMAGE_PAD);
> + if (ret) {
> + unicam_err(unicam, "Unable to register image video
> device.\n");
> + goto unregister;
> + }
> +
> + ret = register_node(unicam, &unicam->node[METADATA_PAD],
> + V4L2_BUF_TYPE_META_CAPTURE, METADATA_PAD);
> + if (ret) {
> + unicam_err(unicam, "Unable to register metadata video
> device.\n");
> + goto unregister;
> + }
> +
> + ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
> + if (ret) {
> + unicam_err(unicam, "Unable to register subdev
> nodes.\n");
> + goto unregister;
> + }
> +
> + /*
> + * Release the initial reference, all references are now owned
> by the
> + * video devices.
> + */
> + unicam_put(unicam);
> + return 0;
> +
> +unregister:
> + unregister_nodes(unicam);
> + unicam_put(unicam);
> +
> + return ret;
> +}
> +
> +static int unicam_async_complete(struct v4l2_async_notifier
> *notifier)
> +{
> + struct unicam_device *unicam = to_unicam_device(notifier-
> >v4l2_dev);
> +
> + return unicam_probe_complete(unicam);
> +}
> +
> +static const struct v4l2_async_notifier_operations unicam_async_ops
> = {
> + .bound = unicam_async_bound,
> + .complete = unicam_async_complete,
> +};
> +
> +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> +{
> + struct platform_device *pdev = dev->pdev;
> + struct v4l2_fwnode_endpoint ep = { 0 };
> + struct device_node *ep_node;
> + struct device_node *sensor_node;
> + unsigned int lane;
> + int ret = -EINVAL;
> +
> + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-
> lanes",
> + &dev->max_data_lanes) < 0) {
> + unicam_err(dev, "number of data lanes not set\n");
> + return -EINVAL;
> + }
> +
> + /* Get the local endpoint and remote device. */
> + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> + if (!ep_node) {
> + unicam_dbg(3, dev, "can't get next endpoint\n");
> + return -EINVAL;
> + }
> +
> + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> +
> + sensor_node = of_graph_get_remote_port_parent(ep_node);
> + if (!sensor_node) {
> + unicam_dbg(3, dev, "can't get remote parent\n");
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> +
> + /* Parse the local endpoint and validate its configuration. */
> + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> +
> + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> + ep.bus_type);
> +
> + dev->bus_type = ep.bus_type;
> +
> + switch (ep.bus_type) {
> + case V4L2_MBUS_CSI2_DPHY:
> + switch (ep.bus.mipi_csi2.num_data_lanes) {
> + case 1:
> + case 2:
> + case 4:
> + break;
> +
> + default:
> + unicam_err(dev, "subdevice %pOF: %u data lanes
> not supported\n",
> + sensor_node,
> + ep.bus.mipi_csi2.num_data_lanes);
> + goto cleanup_exit;
> + }
> +
> + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes;
> lane++) {
> + if (ep.bus.mipi_csi2.data_lanes[lane] != lane +
> 1) {
> + unicam_err(dev, "subdevice %pOF: data
> lanes reordering not supported\n",
> + sensor_node);
> + goto cleanup_exit;
> + }
> + }
> +
> + if (ep.bus.mipi_csi2.num_data_lanes > dev-
> >max_data_lanes) {
> + unicam_err(dev, "subdevice requires %u data
> lanes when %u are supported\n",
> + ep.bus.mipi_csi2.num_data_lanes,
> + dev->max_data_lanes);
> + }
> +
> + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> + dev->bus_flags = ep.bus.mipi_csi2.flags;
> +
> + break;
> +
> + case V4L2_MBUS_CCP2:
> + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> + ep.bus.mipi_csi1.data_lane != 1) {
> + unicam_err(dev, "subdevice %pOF: unsupported
> lanes configuration\n",
> + sensor_node);
> + goto cleanup_exit;
> + }
> +
> + dev->max_data_lanes = 1;
> + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> + break;
> +
> + default:
> + /* Unsupported bus type */
> + unicam_err(dev, "subdevice %pOF: unsupported bus type
> %u\n",
> + sensor_node, ep.bus_type);
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes,
> flags=0x%08x\n",
> + sensor_node,
> + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" :
> "CCP2",
> + dev->max_data_lanes, dev->bus_flags);
> +
> + /* Initialize and register the async notifier. */
> + v4l2_async_notifier_init(&dev->notifier);
> + dev->notifier.ops = &unicam_async_ops;
> +
> + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev-
> >asd);
> + if (ret) {
> + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> + goto cleanup_exit;
> + }
> +
> + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev-
> >notifier);
> + if (ret) {
> + unicam_err(dev, "Error registering async notifier:
> %d\n", ret);
> + ret = -EINVAL;
> + }
> +
> +cleanup_exit:
> + of_node_put(sensor_node);
> + of_node_put(ep_node);
> +
> + return ret;
> +}
> +
> +static int unicam_probe(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam;
> + int ret;
> +
> + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> + if (!unicam)
> + return -ENOMEM;
> +
> + kref_init(&unicam->kref);
> + unicam->pdev = pdev;
> +
> + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(unicam->base)) {
> + unicam_err(unicam, "Failed to get main io block\n");
> + ret = PTR_ERR(unicam->base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev,
> 1);
> + if (IS_ERR(unicam->clk_gate_base)) {
> + unicam_err(unicam, "Failed to get 2nd io block\n");
> + ret = PTR_ERR(unicam->clk_gate_base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> + if (IS_ERR(unicam->clock)) {
> + unicam_err(unicam, "Failed to get clock\n");
> + ret = PTR_ERR(unicam->clock);
> + goto err_unicam_put;
> + }
> +
> + ret = platform_get_irq(pdev, 0);
> + if (ret <= 0) {
> + dev_err(&pdev->dev, "No IRQ resource\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> + "unicam_capture0", unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Unable to request interrupt\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + unicam->mdev.dev = &pdev->dev;
> + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> + sizeof(unicam->mdev.model));
> + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> + "platform:%s", dev_name(&pdev->dev));
> + unicam->mdev.hw_revision = 0;
> +
> + media_device_init(&unicam->mdev);
> +
> + unicam->v4l2_dev.mdev = &unicam->mdev;
> +
> + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> + if (ret) {
> + unicam_err(unicam,
> + "Unable to register v4l2 device.\n");
> + goto err_unicam_put;
> + }
> +
> + ret = media_device_register(&unicam->mdev);
> + if (ret < 0) {
> + unicam_err(unicam,
> + "Unable to register media-controller
> device.\n");
> + goto err_v4l2_unregister;
> + }
> +
> + /* Reserve space for the controls */
> + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> + if (ret < 0)
> + goto err_media_unregister;
> +
> + /* set the driver data in platform device */
> + platform_set_drvdata(pdev, unicam);
> +
> + ret = of_unicam_connect_subdevs(unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> + goto err_media_unregister;
> + }
> +
> + /* Enable the block power domain */
> + pm_runtime_enable(&pdev->dev);
> +
> + return 0;
> +
> +err_media_unregister:
> + media_device_unregister(&unicam->mdev);
> +err_v4l2_unregister:
> + v4l2_device_unregister(&unicam->v4l2_dev);
> +err_unicam_put:
> + unicam_put(unicam);
> +
> + return ret;
> +}
> +
> +static int unicam_remove(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam = platform_get_drvdata(pdev);
> +
> + unicam_dbg(2, unicam, "%s\n", __func__);
> +
> + v4l2_async_notifier_unregister(&unicam->notifier);
> + v4l2_device_unregister(&unicam->v4l2_dev);
> + media_device_unregister(&unicam->mdev);
> + unregister_nodes(unicam);
> +
> + pm_runtime_disable(&pdev->dev);
> +
> + return 0;
> +}
> +
> +static const struct of_device_id unicam_of_match[] = {
> + { .compatible = "brcm,bcm2835-unicam", },
> + { /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> +
> +static struct platform_driver unicam_driver = {
> + .probe = unicam_probe,
> + .remove = unicam_remove,
> + .driver = {
> + .name = UNICAM_MODULE_NAME,
> + .of_match_table = of_match_ptr(unicam_of_match),
> + },
> +};
> +
> +module_platform_driver(unicam_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> +MODULE_LICENSE("GPL");
> +MODULE_VERSION(UNICAM_VERSION);
> diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> new file mode 100644
> index 000000000000..ae059a171d0f
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> @@ -0,0 +1,253 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +/*
> + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + */
> +
> +#ifndef VC4_REGS_UNICAM_H
> +#define VC4_REGS_UNICAM_H
> +
> +/*
> + * The following values are taken from files found within the code
> drop
> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly
> in
> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> + * They have been modified to be only the register offset.
> + */
> +#define UNICAM_CTRL 0x000
> +#define UNICAM_STA 0x004
> +#define UNICAM_ANA 0x008
> +#define UNICAM_PRI 0x00c
> +#define UNICAM_CLK 0x010
> +#define UNICAM_CLT 0x014
> +#define UNICAM_DAT0 0x018
> +#define UNICAM_DAT1 0x01c
> +#define UNICAM_DAT2 0x020
> +#define UNICAM_DAT3 0x024
> +#define UNICAM_DLT 0x028
> +#define UNICAM_CMP0 0x02c
> +#define UNICAM_CMP1 0x030
> +#define UNICAM_CAP0 0x034
> +#define UNICAM_CAP1 0x038
> +#define UNICAM_ICTL 0x100
> +#define UNICAM_ISTA 0x104
> +#define UNICAM_IDI0 0x108
> +#define UNICAM_IPIPE 0x10c
> +#define UNICAM_IBSA0 0x110
> +#define UNICAM_IBEA0 0x114
> +#define UNICAM_IBLS 0x118
> +#define UNICAM_IBWP 0x11c
> +#define UNICAM_IHWIN 0x120
> +#define UNICAM_IHSTA 0x124
> +#define UNICAM_IVWIN 0x128
> +#define UNICAM_IVSTA 0x12c
> +#define UNICAM_ICC 0x130
> +#define UNICAM_ICS 0x134
> +#define UNICAM_IDC 0x138
> +#define UNICAM_IDPO 0x13c
> +#define UNICAM_IDCA 0x140
> +#define UNICAM_IDCD 0x144
> +#define UNICAM_IDS 0x148
> +#define UNICAM_DCS 0x200
> +#define UNICAM_DBSA0 0x204
> +#define UNICAM_DBEA0 0x208
> +#define UNICAM_DBWP 0x20c
> +#define UNICAM_DBCTL 0x300
> +#define UNICAM_IBSA1 0x304
> +#define UNICAM_IBEA1 0x308
> +#define UNICAM_IDI1 0x30c
> +#define UNICAM_DBSA1 0x310
> +#define UNICAM_DBEA1 0x314
> +#define UNICAM_MISC 0x400
> +
> +/*
> + * The following bitmasks are from the kernel released by Broadcom
> + * for Android - https://android.googlesource.com/kernel/bcm/
> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> + * Unicam block as BCM2835, as defined in eg
> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and
> similar.
> + * Values reworked to use the kernel BIT and GENMASK macros.
> + *
> + * Some of the bit mnenomics have been amended to match the
> datasheet.
> + */
> +/* UNICAM_CTRL Register */
> +#define UNICAM_CPE BIT(0)
> +#define UNICAM_MEM BIT(1)
> +#define UNICAM_CPR BIT(2)
> +#define UNICAM_CPM_MASK GENMASK(3, 3)
> +#define UNICAM_CPM_CSI2 0
> +#define UNICAM_CPM_CCP2 1
> +#define UNICAM_SOE BIT(4)
> +#define UNICAM_DCM_MASK GENMASK(5, 5)
> +#define UNICAM_DCM_STROBE 0
> +#define UNICAM_DCM_DATA 1
> +#define UNICAM_SLS BIT(6)
> +#define UNICAM_PFT_MASK GENMASK(11, 8)
> +#define UNICAM_OET_MASK GENMASK(20, 12)
> +
> +/* UNICAM_STA Register */
> +#define UNICAM_SYN BIT(0)
> +#define UNICAM_CS BIT(1)
> +#define UNICAM_SBE BIT(2)
> +#define UNICAM_PBE BIT(3)
> +#define UNICAM_HOE BIT(4)
> +#define UNICAM_PLE BIT(5)
> +#define UNICAM_SSC BIT(6)
> +#define UNICAM_CRCE BIT(7)
> +#define UNICAM_OES BIT(8)
> +#define UNICAM_IFO BIT(9)
> +#define UNICAM_OFO BIT(10)
> +#define UNICAM_BFO BIT(11)
> +#define UNICAM_DL BIT(12)
> +#define UNICAM_PS BIT(13)
> +#define UNICAM_IS BIT(14)
> +#define UNICAM_PI0 BIT(15)
> +#define UNICAM_PI1 BIT(16)
> +#define UNICAM_FSI_S BIT(17)
> +#define UNICAM_FEI_S BIT(18)
> +#define UNICAM_LCI_S BIT(19)
> +#define UNICAM_BUF0_RDY BIT(20)
> +#define UNICAM_BUF0_NO BIT(21)
> +#define UNICAM_BUF1_RDY BIT(22)
> +#define UNICAM_BUF1_NO BIT(23)
> +#define UNICAM_DI BIT(24)
> +
> +#define UNICAM_STA_MASK_ALL \
> + (UNICAM_DL + \
> + UNICAM_SBE + \
> + UNICAM_PBE + \
> + UNICAM_HOE + \
> + UNICAM_PLE + \
> + UNICAM_SSC + \
> + UNICAM_CRCE + \
> + UNICAM_IFO + \
> + UNICAM_OFO + \
> + UNICAM_PS + \
> + UNICAM_PI0 + \
> + UNICAM_PI1)
> +
> +/* UNICAM_ANA Register */
> +#define UNICAM_APD BIT(0)
> +#define UNICAM_BPD BIT(1)
> +#define UNICAM_AR BIT(2)
> +#define UNICAM_DDL BIT(3)
> +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> +
> +/* UNICAM_PRI Register */
> +#define UNICAM_PE BIT(0)
> +#define UNICAM_PT_MASK GENMASK(2, 1)
> +#define UNICAM_NP_MASK GENMASK(7, 4)
> +#define UNICAM_PP_MASK GENMASK(11, 8)
> +#define UNICAM_BS_MASK GENMASK(15, 12)
> +#define UNICAM_BL_MASK GENMASK(17, 16)
> +
> +/* UNICAM_CLK Register */
> +#define UNICAM_CLE BIT(0)
> +#define UNICAM_CLPD BIT(1)
> +#define UNICAM_CLLPE BIT(2)
> +#define UNICAM_CLHSE BIT(3)
> +#define UNICAM_CLTRE BIT(4)
> +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> +#define UNICAM_CLSTE BIT(29)
> +
> +/* UNICAM_CLT Register */
> +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DATn Registers */
> +#define UNICAM_DLE BIT(0)
> +#define UNICAM_DLPD BIT(1)
> +#define UNICAM_DLLPE BIT(2)
> +#define UNICAM_DLHSE BIT(3)
> +#define UNICAM_DLTRE BIT(4)
> +#define UNICAM_DLSM BIT(5)
> +#define UNICAM_DLFO BIT(28)
> +#define UNICAM_DLSTE BIT(29)
> +
> +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> +
> +/* UNICAM_DLT Register */
> +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> +
> +/* UNICAM_ICTL Register */
> +#define UNICAM_FSIE BIT(0)
> +#define UNICAM_FEIE BIT(1)
> +#define UNICAM_IBOB BIT(2)
> +#define UNICAM_FCM BIT(3)
> +#define UNICAM_TFC BIT(4)
> +#define UNICAM_LIP_MASK GENMASK(6, 5)
> +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> +
> +/* UNICAM_IDI0/1 Register */
> +#define UNICAM_ID0_MASK GENMASK(7, 0)
> +#define UNICAM_ID1_MASK GENMASK(15, 8)
> +#define UNICAM_ID2_MASK GENMASK(23, 16)
> +#define UNICAM_ID3_MASK GENMASK(31, 24)
> +
> +/* UNICAM_ISTA Register */
> +#define UNICAM_FSI BIT(0)
> +#define UNICAM_FEI BIT(1)
> +#define UNICAM_LCI BIT(2)
> +
> +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> +
> +/* UNICAM_IPIPE Register */
> +#define UNICAM_PUM_MASK GENMASK(2, 0)
> + /* Unpacking modes */
> + #define UNICAM_PUM_NONE 0
> + #define UNICAM_PUM_UNPACK6 1
> + #define UNICAM_PUM_UNPACK7 2
> + #define UNICAM_PUM_UNPACK8 3
> + #define UNICAM_PUM_UNPACK10 4
> + #define UNICAM_PUM_UNPACK12 5
> + #define UNICAM_PUM_UNPACK14 6
> + #define UNICAM_PUM_UNPACK16 7
> +#define UNICAM_DDM_MASK GENMASK(6, 3)
> +#define UNICAM_PPM_MASK GENMASK(9, 7)
> + /* Packing modes */
> + #define UNICAM_PPM_NONE 0
> + #define UNICAM_PPM_PACK8 1
> + #define UNICAM_PPM_PACK10 2
> + #define UNICAM_PPM_PACK12 3
> + #define UNICAM_PPM_PACK14 4
> + #define UNICAM_PPM_PACK16 5
> +#define UNICAM_DEM_MASK GENMASK(11, 10)
> +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> +#define UNICAM_ICM_MASK GENMASK(16, 15)
> +#define UNICAM_IDM_MASK GENMASK(17, 17)
> +
> +/* UNICAM_ICC Register */
> +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> +#define UNICAM_ICST_MASK GENMASK(12, 10)
> +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> +
> +/* UNICAM_DCS Register */
> +#define UNICAM_DIE BIT(0)
> +#define UNICAM_DIM BIT(1)
> +#define UNICAM_DBOB BIT(3)
> +#define UNICAM_FDE BIT(4)
> +#define UNICAM_LDP BIT(5)
> +#define UNICAM_EDL_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DBCTL Register */
> +#define UNICAM_DBEN BIT(0)
> +#define UNICAM_BUF0_IE BIT(1)
> +#define UNICAM_BUF1_IE BIT(2)
> +
> +/* UNICAM_CMP[0,1] register */
> +#define UNICAM_PCE BIT(31)
> +#define UNICAM_GI BIT(9)
> +#define UNICAM_CPH BIT(8)
> +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> +
> +/* UNICAM_MISC register */
> +#define UNICAM_FL0 BIT(6)
> +#define UNICAM_FL1 BIT(9)
> +
> +#endif
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-05-04 9:25 ` [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface Laurent Pinchart
` (2 preceding siblings ...)
2020-05-06 18:01 ` Nicolas Saenz Julienne
@ 2020-08-29 11:20 ` Jacopo Mondi
2020-08-29 18:32 ` Laurent Pinchart
2020-09-15 7:03 ` Sakari Ailus
4 siblings, 1 reply; 111+ messages in thread
From: Jacopo Mondi @ 2020-08-29 11:20 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, Kieran Bingham, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Laurent, Dave, Naush,
On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Add a driver for the Unicam camera receiver block on BCM283x processors.
> Compared to the bcm2835-camera driver present in staging, this driver
> handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> the VC4 firmware running on the VPU.
>
> The commit is made up of a series of changes cherry-picked from the
> rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> additional enhancements, forward-ported to the mainline kernel.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v1:
>
> - Re-fetch mbus code from subdev on a g_fmt call
> - Group all ioctl disabling together
> - Fix reference counting in unicam_open
> - Add support for VIDIOC_[S|G]_SELECTION
> ---
[snip]
> +
> +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> +{
> + struct platform_device *pdev = dev->pdev;
> + struct v4l2_fwnode_endpoint ep = { 0 };
> + struct device_node *ep_node;
> + struct device_node *sensor_node;
> + unsigned int lane;
> + int ret = -EINVAL;
> +
> + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> + &dev->max_data_lanes) < 0) {
> + unicam_err(dev, "number of data lanes not set\n");
> + return -EINVAL;
> + }
mmm, this seems like a good use case for the newly introduced
get_mbus_config pad operation, isn't it ?
We could drop the custom "brcm,num-data-lanes" property, use the
standard "num-lanes" to describe the number of wired data lanes, and
get the number of actually used ones at s_stream(1) time with the new pad
operation
What do you think ? This would require implemeting the operation in
each subdev the UNICAM interface has to work with, but this doesn't
sound too much of a work.
For reference this is how it happens on rcar+adv748x:
https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
Thanks
j
> +
> + /* Get the local endpoint and remote device. */
> + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> + if (!ep_node) {
> + unicam_dbg(3, dev, "can't get next endpoint\n");
> + return -EINVAL;
> + }
> +
> + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> +
> + sensor_node = of_graph_get_remote_port_parent(ep_node);
> + if (!sensor_node) {
> + unicam_dbg(3, dev, "can't get remote parent\n");
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> +
> + /* Parse the local endpoint and validate its configuration. */
> + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> +
> + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> + ep.bus_type);
> +
> + dev->bus_type = ep.bus_type;
> +
> + switch (ep.bus_type) {
> + case V4L2_MBUS_CSI2_DPHY:
> + switch (ep.bus.mipi_csi2.num_data_lanes) {
> + case 1:
> + case 2:
> + case 4:
> + break;
> +
> + default:
> + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> + sensor_node,
> + ep.bus.mipi_csi2.num_data_lanes);
> + goto cleanup_exit;
> + }
> +
> + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> + sensor_node);
> + goto cleanup_exit;
> + }
> + }
> +
> + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> + ep.bus.mipi_csi2.num_data_lanes,
> + dev->max_data_lanes);
> + }
> +
> + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> + dev->bus_flags = ep.bus.mipi_csi2.flags;
> +
> + break;
> +
> + case V4L2_MBUS_CCP2:
> + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> + ep.bus.mipi_csi1.data_lane != 1) {
> + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> + sensor_node);
> + goto cleanup_exit;
> + }
> +
> + dev->max_data_lanes = 1;
> + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> + break;
> +
> + default:
> + /* Unsupported bus type */
> + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> + sensor_node, ep.bus_type);
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> + sensor_node,
> + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> + dev->max_data_lanes, dev->bus_flags);
> +
> + /* Initialize and register the async notifier. */
> + v4l2_async_notifier_init(&dev->notifier);
> + dev->notifier.ops = &unicam_async_ops;
> +
> + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> + if (ret) {
> + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> + goto cleanup_exit;
> + }
> +
> + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> + if (ret) {
> + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> + ret = -EINVAL;
> + }
> +
> +cleanup_exit:
> + of_node_put(sensor_node);
> + of_node_put(ep_node);
> +
> + return ret;
> +}
> +
> +static int unicam_probe(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam;
> + int ret;
> +
> + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> + if (!unicam)
> + return -ENOMEM;
> +
> + kref_init(&unicam->kref);
> + unicam->pdev = pdev;
> +
> + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> + if (IS_ERR(unicam->base)) {
> + unicam_err(unicam, "Failed to get main io block\n");
> + ret = PTR_ERR(unicam->base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> + if (IS_ERR(unicam->clk_gate_base)) {
> + unicam_err(unicam, "Failed to get 2nd io block\n");
> + ret = PTR_ERR(unicam->clk_gate_base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> + if (IS_ERR(unicam->clock)) {
> + unicam_err(unicam, "Failed to get clock\n");
> + ret = PTR_ERR(unicam->clock);
> + goto err_unicam_put;
> + }
> +
> + ret = platform_get_irq(pdev, 0);
> + if (ret <= 0) {
> + dev_err(&pdev->dev, "No IRQ resource\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> + "unicam_capture0", unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Unable to request interrupt\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + unicam->mdev.dev = &pdev->dev;
> + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> + sizeof(unicam->mdev.model));
> + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> + "platform:%s", dev_name(&pdev->dev));
> + unicam->mdev.hw_revision = 0;
> +
> + media_device_init(&unicam->mdev);
> +
> + unicam->v4l2_dev.mdev = &unicam->mdev;
> +
> + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> + if (ret) {
> + unicam_err(unicam,
> + "Unable to register v4l2 device.\n");
> + goto err_unicam_put;
> + }
> +
> + ret = media_device_register(&unicam->mdev);
> + if (ret < 0) {
> + unicam_err(unicam,
> + "Unable to register media-controller device.\n");
> + goto err_v4l2_unregister;
> + }
> +
> + /* Reserve space for the controls */
> + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> + if (ret < 0)
> + goto err_media_unregister;
> +
> + /* set the driver data in platform device */
> + platform_set_drvdata(pdev, unicam);
> +
> + ret = of_unicam_connect_subdevs(unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> + goto err_media_unregister;
> + }
> +
> + /* Enable the block power domain */
> + pm_runtime_enable(&pdev->dev);
> +
> + return 0;
> +
> +err_media_unregister:
> + media_device_unregister(&unicam->mdev);
> +err_v4l2_unregister:
> + v4l2_device_unregister(&unicam->v4l2_dev);
> +err_unicam_put:
> + unicam_put(unicam);
> +
> + return ret;
> +}
> +
> +static int unicam_remove(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam = platform_get_drvdata(pdev);
> +
> + unicam_dbg(2, unicam, "%s\n", __func__);
> +
> + v4l2_async_notifier_unregister(&unicam->notifier);
> + v4l2_device_unregister(&unicam->v4l2_dev);
> + media_device_unregister(&unicam->mdev);
> + unregister_nodes(unicam);
> +
> + pm_runtime_disable(&pdev->dev);
> +
> + return 0;
> +}
> +
> +static const struct of_device_id unicam_of_match[] = {
> + { .compatible = "brcm,bcm2835-unicam", },
> + { /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> +
> +static struct platform_driver unicam_driver = {
> + .probe = unicam_probe,
> + .remove = unicam_remove,
> + .driver = {
> + .name = UNICAM_MODULE_NAME,
> + .of_match_table = of_match_ptr(unicam_of_match),
> + },
> +};
> +
> +module_platform_driver(unicam_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> +MODULE_LICENSE("GPL");
> +MODULE_VERSION(UNICAM_VERSION);
> diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> new file mode 100644
> index 000000000000..ae059a171d0f
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> @@ -0,0 +1,253 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +/*
> + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + */
> +
> +#ifndef VC4_REGS_UNICAM_H
> +#define VC4_REGS_UNICAM_H
> +
> +/*
> + * The following values are taken from files found within the code drop
> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> + * They have been modified to be only the register offset.
> + */
> +#define UNICAM_CTRL 0x000
> +#define UNICAM_STA 0x004
> +#define UNICAM_ANA 0x008
> +#define UNICAM_PRI 0x00c
> +#define UNICAM_CLK 0x010
> +#define UNICAM_CLT 0x014
> +#define UNICAM_DAT0 0x018
> +#define UNICAM_DAT1 0x01c
> +#define UNICAM_DAT2 0x020
> +#define UNICAM_DAT3 0x024
> +#define UNICAM_DLT 0x028
> +#define UNICAM_CMP0 0x02c
> +#define UNICAM_CMP1 0x030
> +#define UNICAM_CAP0 0x034
> +#define UNICAM_CAP1 0x038
> +#define UNICAM_ICTL 0x100
> +#define UNICAM_ISTA 0x104
> +#define UNICAM_IDI0 0x108
> +#define UNICAM_IPIPE 0x10c
> +#define UNICAM_IBSA0 0x110
> +#define UNICAM_IBEA0 0x114
> +#define UNICAM_IBLS 0x118
> +#define UNICAM_IBWP 0x11c
> +#define UNICAM_IHWIN 0x120
> +#define UNICAM_IHSTA 0x124
> +#define UNICAM_IVWIN 0x128
> +#define UNICAM_IVSTA 0x12c
> +#define UNICAM_ICC 0x130
> +#define UNICAM_ICS 0x134
> +#define UNICAM_IDC 0x138
> +#define UNICAM_IDPO 0x13c
> +#define UNICAM_IDCA 0x140
> +#define UNICAM_IDCD 0x144
> +#define UNICAM_IDS 0x148
> +#define UNICAM_DCS 0x200
> +#define UNICAM_DBSA0 0x204
> +#define UNICAM_DBEA0 0x208
> +#define UNICAM_DBWP 0x20c
> +#define UNICAM_DBCTL 0x300
> +#define UNICAM_IBSA1 0x304
> +#define UNICAM_IBEA1 0x308
> +#define UNICAM_IDI1 0x30c
> +#define UNICAM_DBSA1 0x310
> +#define UNICAM_DBEA1 0x314
> +#define UNICAM_MISC 0x400
> +
> +/*
> + * The following bitmasks are from the kernel released by Broadcom
> + * for Android - https://android.googlesource.com/kernel/bcm/
> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> + * Unicam block as BCM2835, as defined in eg
> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> + * Values reworked to use the kernel BIT and GENMASK macros.
> + *
> + * Some of the bit mnenomics have been amended to match the datasheet.
> + */
> +/* UNICAM_CTRL Register */
> +#define UNICAM_CPE BIT(0)
> +#define UNICAM_MEM BIT(1)
> +#define UNICAM_CPR BIT(2)
> +#define UNICAM_CPM_MASK GENMASK(3, 3)
> +#define UNICAM_CPM_CSI2 0
> +#define UNICAM_CPM_CCP2 1
> +#define UNICAM_SOE BIT(4)
> +#define UNICAM_DCM_MASK GENMASK(5, 5)
> +#define UNICAM_DCM_STROBE 0
> +#define UNICAM_DCM_DATA 1
> +#define UNICAM_SLS BIT(6)
> +#define UNICAM_PFT_MASK GENMASK(11, 8)
> +#define UNICAM_OET_MASK GENMASK(20, 12)
> +
> +/* UNICAM_STA Register */
> +#define UNICAM_SYN BIT(0)
> +#define UNICAM_CS BIT(1)
> +#define UNICAM_SBE BIT(2)
> +#define UNICAM_PBE BIT(3)
> +#define UNICAM_HOE BIT(4)
> +#define UNICAM_PLE BIT(5)
> +#define UNICAM_SSC BIT(6)
> +#define UNICAM_CRCE BIT(7)
> +#define UNICAM_OES BIT(8)
> +#define UNICAM_IFO BIT(9)
> +#define UNICAM_OFO BIT(10)
> +#define UNICAM_BFO BIT(11)
> +#define UNICAM_DL BIT(12)
> +#define UNICAM_PS BIT(13)
> +#define UNICAM_IS BIT(14)
> +#define UNICAM_PI0 BIT(15)
> +#define UNICAM_PI1 BIT(16)
> +#define UNICAM_FSI_S BIT(17)
> +#define UNICAM_FEI_S BIT(18)
> +#define UNICAM_LCI_S BIT(19)
> +#define UNICAM_BUF0_RDY BIT(20)
> +#define UNICAM_BUF0_NO BIT(21)
> +#define UNICAM_BUF1_RDY BIT(22)
> +#define UNICAM_BUF1_NO BIT(23)
> +#define UNICAM_DI BIT(24)
> +
> +#define UNICAM_STA_MASK_ALL \
> + (UNICAM_DL + \
> + UNICAM_SBE + \
> + UNICAM_PBE + \
> + UNICAM_HOE + \
> + UNICAM_PLE + \
> + UNICAM_SSC + \
> + UNICAM_CRCE + \
> + UNICAM_IFO + \
> + UNICAM_OFO + \
> + UNICAM_PS + \
> + UNICAM_PI0 + \
> + UNICAM_PI1)
> +
> +/* UNICAM_ANA Register */
> +#define UNICAM_APD BIT(0)
> +#define UNICAM_BPD BIT(1)
> +#define UNICAM_AR BIT(2)
> +#define UNICAM_DDL BIT(3)
> +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> +
> +/* UNICAM_PRI Register */
> +#define UNICAM_PE BIT(0)
> +#define UNICAM_PT_MASK GENMASK(2, 1)
> +#define UNICAM_NP_MASK GENMASK(7, 4)
> +#define UNICAM_PP_MASK GENMASK(11, 8)
> +#define UNICAM_BS_MASK GENMASK(15, 12)
> +#define UNICAM_BL_MASK GENMASK(17, 16)
> +
> +/* UNICAM_CLK Register */
> +#define UNICAM_CLE BIT(0)
> +#define UNICAM_CLPD BIT(1)
> +#define UNICAM_CLLPE BIT(2)
> +#define UNICAM_CLHSE BIT(3)
> +#define UNICAM_CLTRE BIT(4)
> +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> +#define UNICAM_CLSTE BIT(29)
> +
> +/* UNICAM_CLT Register */
> +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DATn Registers */
> +#define UNICAM_DLE BIT(0)
> +#define UNICAM_DLPD BIT(1)
> +#define UNICAM_DLLPE BIT(2)
> +#define UNICAM_DLHSE BIT(3)
> +#define UNICAM_DLTRE BIT(4)
> +#define UNICAM_DLSM BIT(5)
> +#define UNICAM_DLFO BIT(28)
> +#define UNICAM_DLSTE BIT(29)
> +
> +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> +
> +/* UNICAM_DLT Register */
> +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> +
> +/* UNICAM_ICTL Register */
> +#define UNICAM_FSIE BIT(0)
> +#define UNICAM_FEIE BIT(1)
> +#define UNICAM_IBOB BIT(2)
> +#define UNICAM_FCM BIT(3)
> +#define UNICAM_TFC BIT(4)
> +#define UNICAM_LIP_MASK GENMASK(6, 5)
> +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> +
> +/* UNICAM_IDI0/1 Register */
> +#define UNICAM_ID0_MASK GENMASK(7, 0)
> +#define UNICAM_ID1_MASK GENMASK(15, 8)
> +#define UNICAM_ID2_MASK GENMASK(23, 16)
> +#define UNICAM_ID3_MASK GENMASK(31, 24)
> +
> +/* UNICAM_ISTA Register */
> +#define UNICAM_FSI BIT(0)
> +#define UNICAM_FEI BIT(1)
> +#define UNICAM_LCI BIT(2)
> +
> +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> +
> +/* UNICAM_IPIPE Register */
> +#define UNICAM_PUM_MASK GENMASK(2, 0)
> + /* Unpacking modes */
> + #define UNICAM_PUM_NONE 0
> + #define UNICAM_PUM_UNPACK6 1
> + #define UNICAM_PUM_UNPACK7 2
> + #define UNICAM_PUM_UNPACK8 3
> + #define UNICAM_PUM_UNPACK10 4
> + #define UNICAM_PUM_UNPACK12 5
> + #define UNICAM_PUM_UNPACK14 6
> + #define UNICAM_PUM_UNPACK16 7
> +#define UNICAM_DDM_MASK GENMASK(6, 3)
> +#define UNICAM_PPM_MASK GENMASK(9, 7)
> + /* Packing modes */
> + #define UNICAM_PPM_NONE 0
> + #define UNICAM_PPM_PACK8 1
> + #define UNICAM_PPM_PACK10 2
> + #define UNICAM_PPM_PACK12 3
> + #define UNICAM_PPM_PACK14 4
> + #define UNICAM_PPM_PACK16 5
> +#define UNICAM_DEM_MASK GENMASK(11, 10)
> +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> +#define UNICAM_ICM_MASK GENMASK(16, 15)
> +#define UNICAM_IDM_MASK GENMASK(17, 17)
> +
> +/* UNICAM_ICC Register */
> +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> +#define UNICAM_ICST_MASK GENMASK(12, 10)
> +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> +
> +/* UNICAM_DCS Register */
> +#define UNICAM_DIE BIT(0)
> +#define UNICAM_DIM BIT(1)
> +#define UNICAM_DBOB BIT(3)
> +#define UNICAM_FDE BIT(4)
> +#define UNICAM_LDP BIT(5)
> +#define UNICAM_EDL_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DBCTL Register */
> +#define UNICAM_DBEN BIT(0)
> +#define UNICAM_BUF0_IE BIT(1)
> +#define UNICAM_BUF1_IE BIT(2)
> +
> +/* UNICAM_CMP[0,1] register */
> +#define UNICAM_PCE BIT(31)
> +#define UNICAM_GI BIT(9)
> +#define UNICAM_CPH BIT(8)
> +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> +
> +/* UNICAM_MISC register */
> +#define UNICAM_FL0 BIT(6)
> +#define UNICAM_FL1 BIT(9)
> +
> +#endif
> --
> Regards,
>
> Laurent Pinchart
>
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-08-29 11:20 ` Jacopo Mondi
@ 2020-08-29 18:32 ` Laurent Pinchart
2020-08-31 7:38 ` Jacopo Mondi
0 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-08-29 18:32 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, Kieran Bingham, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Jacopo,
On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> Hi Laurent, Dave, Naush,
>
> On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > Compared to the bcm2835-camera driver present in staging, this driver
> > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > the VC4 firmware running on the VPU.
> >
> > The commit is made up of a series of changes cherry-picked from the
> > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > additional enhancements, forward-ported to the mainline kernel.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v1:
> >
> > - Re-fetch mbus code from subdev on a g_fmt call
> > - Group all ioctl disabling together
> > - Fix reference counting in unicam_open
> > - Add support for VIDIOC_[S|G]_SELECTION
> > ---
>
> [snip]
>
> > +
> > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > +{
> > + struct platform_device *pdev = dev->pdev;
> > + struct v4l2_fwnode_endpoint ep = { 0 };
> > + struct device_node *ep_node;
> > + struct device_node *sensor_node;
> > + unsigned int lane;
> > + int ret = -EINVAL;
> > +
> > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > + &dev->max_data_lanes) < 0) {
> > + unicam_err(dev, "number of data lanes not set\n");
> > + return -EINVAL;
> > + }
>
> mmm, this seems like a good use case for the newly introduced
> get_mbus_config pad operation, isn't it ?
>
> We could drop the custom "brcm,num-data-lanes" property, use the
> standard "num-lanes" to describe the number of wired data lanes, and
> get the number of actually used ones at s_stream(1) time with the new pad
> operation
>
> What do you think ? This would require implemeting the operation in
> each subdev the UNICAM interface has to work with, but this doesn't
> sound too much of a work.
>
> For reference this is how it happens on rcar+adv748x:
> https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
The former tells how many data lanes the unicam IP has (the same IP,
with the same compatible string, is instantiated in different SoCs with
different number of lanes), while the latter queries at runtime how many
data lanes the remote subdev will use for its current configuration. The
DT property serves as a sanity check, to ensure to no more than the
number of supported data lanes is used, while the subdev operation
should be used in the unicam driver to support sources that will
transmit over a different number of data lanes depending on the
configuration (mostly depending on the resolution). We don't have to
implement support for get_mbus_config just yet, it can be delayed until
we have a source that requires it.
> > +
> > + /* Get the local endpoint and remote device. */
> > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > + if (!ep_node) {
> > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > + return -EINVAL;
> > + }
> > +
> > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > +
> > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > + if (!sensor_node) {
> > + unicam_dbg(3, dev, "can't get remote parent\n");
> > + goto cleanup_exit;
> > + }
> > +
> > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > +
> > + /* Parse the local endpoint and validate its configuration. */
> > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > +
> > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > + ep.bus_type);
> > +
> > + dev->bus_type = ep.bus_type;
> > +
> > + switch (ep.bus_type) {
> > + case V4L2_MBUS_CSI2_DPHY:
> > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > + case 1:
> > + case 2:
> > + case 4:
> > + break;
> > +
> > + default:
> > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > + sensor_node,
> > + ep.bus.mipi_csi2.num_data_lanes);
> > + goto cleanup_exit;
> > + }
> > +
> > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > + sensor_node);
> > + goto cleanup_exit;
> > + }
> > + }
> > +
> > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > + ep.bus.mipi_csi2.num_data_lanes,
> > + dev->max_data_lanes);
> > + }
> > +
> > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > +
> > + break;
> > +
> > + case V4L2_MBUS_CCP2:
> > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > + ep.bus.mipi_csi1.data_lane != 1) {
> > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > + sensor_node);
> > + goto cleanup_exit;
> > + }
> > +
> > + dev->max_data_lanes = 1;
> > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > + break;
> > +
> > + default:
> > + /* Unsupported bus type */
> > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > + sensor_node, ep.bus_type);
> > + goto cleanup_exit;
> > + }
> > +
> > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > + sensor_node,
> > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > + dev->max_data_lanes, dev->bus_flags);
> > +
> > + /* Initialize and register the async notifier. */
> > + v4l2_async_notifier_init(&dev->notifier);
> > + dev->notifier.ops = &unicam_async_ops;
> > +
> > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > + if (ret) {
> > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > + goto cleanup_exit;
> > + }
> > +
> > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > + if (ret) {
> > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > + ret = -EINVAL;
> > + }
> > +
> > +cleanup_exit:
> > + of_node_put(sensor_node);
> > + of_node_put(ep_node);
> > +
> > + return ret;
> > +}
> > +
> > +static int unicam_probe(struct platform_device *pdev)
> > +{
> > + struct unicam_device *unicam;
> > + int ret;
> > +
> > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > + if (!unicam)
> > + return -ENOMEM;
> > +
> > + kref_init(&unicam->kref);
> > + unicam->pdev = pdev;
> > +
> > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > + if (IS_ERR(unicam->base)) {
> > + unicam_err(unicam, "Failed to get main io block\n");
> > + ret = PTR_ERR(unicam->base);
> > + goto err_unicam_put;
> > + }
> > +
> > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > + if (IS_ERR(unicam->clk_gate_base)) {
> > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > + ret = PTR_ERR(unicam->clk_gate_base);
> > + goto err_unicam_put;
> > + }
> > +
> > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > + if (IS_ERR(unicam->clock)) {
> > + unicam_err(unicam, "Failed to get clock\n");
> > + ret = PTR_ERR(unicam->clock);
> > + goto err_unicam_put;
> > + }
> > +
> > + ret = platform_get_irq(pdev, 0);
> > + if (ret <= 0) {
> > + dev_err(&pdev->dev, "No IRQ resource\n");
> > + ret = -EINVAL;
> > + goto err_unicam_put;
> > + }
> > +
> > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > + "unicam_capture0", unicam);
> > + if (ret) {
> > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > + ret = -EINVAL;
> > + goto err_unicam_put;
> > + }
> > +
> > + unicam->mdev.dev = &pdev->dev;
> > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > + sizeof(unicam->mdev.model));
> > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > + "platform:%s", dev_name(&pdev->dev));
> > + unicam->mdev.hw_revision = 0;
> > +
> > + media_device_init(&unicam->mdev);
> > +
> > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > +
> > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > + if (ret) {
> > + unicam_err(unicam,
> > + "Unable to register v4l2 device.\n");
> > + goto err_unicam_put;
> > + }
> > +
> > + ret = media_device_register(&unicam->mdev);
> > + if (ret < 0) {
> > + unicam_err(unicam,
> > + "Unable to register media-controller device.\n");
> > + goto err_v4l2_unregister;
> > + }
> > +
> > + /* Reserve space for the controls */
> > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > + if (ret < 0)
> > + goto err_media_unregister;
> > +
> > + /* set the driver data in platform device */
> > + platform_set_drvdata(pdev, unicam);
> > +
> > + ret = of_unicam_connect_subdevs(unicam);
> > + if (ret) {
> > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > + goto err_media_unregister;
> > + }
> > +
> > + /* Enable the block power domain */
> > + pm_runtime_enable(&pdev->dev);
> > +
> > + return 0;
> > +
> > +err_media_unregister:
> > + media_device_unregister(&unicam->mdev);
> > +err_v4l2_unregister:
> > + v4l2_device_unregister(&unicam->v4l2_dev);
> > +err_unicam_put:
> > + unicam_put(unicam);
> > +
> > + return ret;
> > +}
> > +
> > +static int unicam_remove(struct platform_device *pdev)
> > +{
> > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > +
> > + unicam_dbg(2, unicam, "%s\n", __func__);
> > +
> > + v4l2_async_notifier_unregister(&unicam->notifier);
> > + v4l2_device_unregister(&unicam->v4l2_dev);
> > + media_device_unregister(&unicam->mdev);
> > + unregister_nodes(unicam);
> > +
> > + pm_runtime_disable(&pdev->dev);
> > +
> > + return 0;
> > +}
> > +
> > +static const struct of_device_id unicam_of_match[] = {
> > + { .compatible = "brcm,bcm2835-unicam", },
> > + { /* sentinel */ },
> > +};
> > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > +
> > +static struct platform_driver unicam_driver = {
> > + .probe = unicam_probe,
> > + .remove = unicam_remove,
> > + .driver = {
> > + .name = UNICAM_MODULE_NAME,
> > + .of_match_table = of_match_ptr(unicam_of_match),
> > + },
> > +};
> > +
> > +module_platform_driver(unicam_driver);
> > +
> > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > +MODULE_LICENSE("GPL");
> > +MODULE_VERSION(UNICAM_VERSION);
> > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > new file mode 100644
> > index 000000000000..ae059a171d0f
> > --- /dev/null
> > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > @@ -0,0 +1,253 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +
> > +/*
> > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > + */
> > +
> > +#ifndef VC4_REGS_UNICAM_H
> > +#define VC4_REGS_UNICAM_H
> > +
> > +/*
> > + * The following values are taken from files found within the code drop
> > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > + * They have been modified to be only the register offset.
> > + */
> > +#define UNICAM_CTRL 0x000
> > +#define UNICAM_STA 0x004
> > +#define UNICAM_ANA 0x008
> > +#define UNICAM_PRI 0x00c
> > +#define UNICAM_CLK 0x010
> > +#define UNICAM_CLT 0x014
> > +#define UNICAM_DAT0 0x018
> > +#define UNICAM_DAT1 0x01c
> > +#define UNICAM_DAT2 0x020
> > +#define UNICAM_DAT3 0x024
> > +#define UNICAM_DLT 0x028
> > +#define UNICAM_CMP0 0x02c
> > +#define UNICAM_CMP1 0x030
> > +#define UNICAM_CAP0 0x034
> > +#define UNICAM_CAP1 0x038
> > +#define UNICAM_ICTL 0x100
> > +#define UNICAM_ISTA 0x104
> > +#define UNICAM_IDI0 0x108
> > +#define UNICAM_IPIPE 0x10c
> > +#define UNICAM_IBSA0 0x110
> > +#define UNICAM_IBEA0 0x114
> > +#define UNICAM_IBLS 0x118
> > +#define UNICAM_IBWP 0x11c
> > +#define UNICAM_IHWIN 0x120
> > +#define UNICAM_IHSTA 0x124
> > +#define UNICAM_IVWIN 0x128
> > +#define UNICAM_IVSTA 0x12c
> > +#define UNICAM_ICC 0x130
> > +#define UNICAM_ICS 0x134
> > +#define UNICAM_IDC 0x138
> > +#define UNICAM_IDPO 0x13c
> > +#define UNICAM_IDCA 0x140
> > +#define UNICAM_IDCD 0x144
> > +#define UNICAM_IDS 0x148
> > +#define UNICAM_DCS 0x200
> > +#define UNICAM_DBSA0 0x204
> > +#define UNICAM_DBEA0 0x208
> > +#define UNICAM_DBWP 0x20c
> > +#define UNICAM_DBCTL 0x300
> > +#define UNICAM_IBSA1 0x304
> > +#define UNICAM_IBEA1 0x308
> > +#define UNICAM_IDI1 0x30c
> > +#define UNICAM_DBSA1 0x310
> > +#define UNICAM_DBEA1 0x314
> > +#define UNICAM_MISC 0x400
> > +
> > +/*
> > + * The following bitmasks are from the kernel released by Broadcom
> > + * for Android - https://android.googlesource.com/kernel/bcm/
> > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > + * Unicam block as BCM2835, as defined in eg
> > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > + * Values reworked to use the kernel BIT and GENMASK macros.
> > + *
> > + * Some of the bit mnenomics have been amended to match the datasheet.
> > + */
> > +/* UNICAM_CTRL Register */
> > +#define UNICAM_CPE BIT(0)
> > +#define UNICAM_MEM BIT(1)
> > +#define UNICAM_CPR BIT(2)
> > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > +#define UNICAM_CPM_CSI2 0
> > +#define UNICAM_CPM_CCP2 1
> > +#define UNICAM_SOE BIT(4)
> > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > +#define UNICAM_DCM_STROBE 0
> > +#define UNICAM_DCM_DATA 1
> > +#define UNICAM_SLS BIT(6)
> > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > +
> > +/* UNICAM_STA Register */
> > +#define UNICAM_SYN BIT(0)
> > +#define UNICAM_CS BIT(1)
> > +#define UNICAM_SBE BIT(2)
> > +#define UNICAM_PBE BIT(3)
> > +#define UNICAM_HOE BIT(4)
> > +#define UNICAM_PLE BIT(5)
> > +#define UNICAM_SSC BIT(6)
> > +#define UNICAM_CRCE BIT(7)
> > +#define UNICAM_OES BIT(8)
> > +#define UNICAM_IFO BIT(9)
> > +#define UNICAM_OFO BIT(10)
> > +#define UNICAM_BFO BIT(11)
> > +#define UNICAM_DL BIT(12)
> > +#define UNICAM_PS BIT(13)
> > +#define UNICAM_IS BIT(14)
> > +#define UNICAM_PI0 BIT(15)
> > +#define UNICAM_PI1 BIT(16)
> > +#define UNICAM_FSI_S BIT(17)
> > +#define UNICAM_FEI_S BIT(18)
> > +#define UNICAM_LCI_S BIT(19)
> > +#define UNICAM_BUF0_RDY BIT(20)
> > +#define UNICAM_BUF0_NO BIT(21)
> > +#define UNICAM_BUF1_RDY BIT(22)
> > +#define UNICAM_BUF1_NO BIT(23)
> > +#define UNICAM_DI BIT(24)
> > +
> > +#define UNICAM_STA_MASK_ALL \
> > + (UNICAM_DL + \
> > + UNICAM_SBE + \
> > + UNICAM_PBE + \
> > + UNICAM_HOE + \
> > + UNICAM_PLE + \
> > + UNICAM_SSC + \
> > + UNICAM_CRCE + \
> > + UNICAM_IFO + \
> > + UNICAM_OFO + \
> > + UNICAM_PS + \
> > + UNICAM_PI0 + \
> > + UNICAM_PI1)
> > +
> > +/* UNICAM_ANA Register */
> > +#define UNICAM_APD BIT(0)
> > +#define UNICAM_BPD BIT(1)
> > +#define UNICAM_AR BIT(2)
> > +#define UNICAM_DDL BIT(3)
> > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > +
> > +/* UNICAM_PRI Register */
> > +#define UNICAM_PE BIT(0)
> > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > +
> > +/* UNICAM_CLK Register */
> > +#define UNICAM_CLE BIT(0)
> > +#define UNICAM_CLPD BIT(1)
> > +#define UNICAM_CLLPE BIT(2)
> > +#define UNICAM_CLHSE BIT(3)
> > +#define UNICAM_CLTRE BIT(4)
> > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > +#define UNICAM_CLSTE BIT(29)
> > +
> > +/* UNICAM_CLT Register */
> > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > +
> > +/* UNICAM_DATn Registers */
> > +#define UNICAM_DLE BIT(0)
> > +#define UNICAM_DLPD BIT(1)
> > +#define UNICAM_DLLPE BIT(2)
> > +#define UNICAM_DLHSE BIT(3)
> > +#define UNICAM_DLTRE BIT(4)
> > +#define UNICAM_DLSM BIT(5)
> > +#define UNICAM_DLFO BIT(28)
> > +#define UNICAM_DLSTE BIT(29)
> > +
> > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > +
> > +/* UNICAM_DLT Register */
> > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > +
> > +/* UNICAM_ICTL Register */
> > +#define UNICAM_FSIE BIT(0)
> > +#define UNICAM_FEIE BIT(1)
> > +#define UNICAM_IBOB BIT(2)
> > +#define UNICAM_FCM BIT(3)
> > +#define UNICAM_TFC BIT(4)
> > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > +
> > +/* UNICAM_IDI0/1 Register */
> > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > +
> > +/* UNICAM_ISTA Register */
> > +#define UNICAM_FSI BIT(0)
> > +#define UNICAM_FEI BIT(1)
> > +#define UNICAM_LCI BIT(2)
> > +
> > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > +
> > +/* UNICAM_IPIPE Register */
> > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > + /* Unpacking modes */
> > + #define UNICAM_PUM_NONE 0
> > + #define UNICAM_PUM_UNPACK6 1
> > + #define UNICAM_PUM_UNPACK7 2
> > + #define UNICAM_PUM_UNPACK8 3
> > + #define UNICAM_PUM_UNPACK10 4
> > + #define UNICAM_PUM_UNPACK12 5
> > + #define UNICAM_PUM_UNPACK14 6
> > + #define UNICAM_PUM_UNPACK16 7
> > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > + /* Packing modes */
> > + #define UNICAM_PPM_NONE 0
> > + #define UNICAM_PPM_PACK8 1
> > + #define UNICAM_PPM_PACK10 2
> > + #define UNICAM_PPM_PACK12 3
> > + #define UNICAM_PPM_PACK14 4
> > + #define UNICAM_PPM_PACK16 5
> > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > +
> > +/* UNICAM_ICC Register */
> > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > +
> > +/* UNICAM_DCS Register */
> > +#define UNICAM_DIE BIT(0)
> > +#define UNICAM_DIM BIT(1)
> > +#define UNICAM_DBOB BIT(3)
> > +#define UNICAM_FDE BIT(4)
> > +#define UNICAM_LDP BIT(5)
> > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > +
> > +/* UNICAM_DBCTL Register */
> > +#define UNICAM_DBEN BIT(0)
> > +#define UNICAM_BUF0_IE BIT(1)
> > +#define UNICAM_BUF1_IE BIT(2)
> > +
> > +/* UNICAM_CMP[0,1] register */
> > +#define UNICAM_PCE BIT(31)
> > +#define UNICAM_GI BIT(9)
> > +#define UNICAM_CPH BIT(8)
> > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > +
> > +/* UNICAM_MISC register */
> > +#define UNICAM_FL0 BIT(6)
> > +#define UNICAM_FL1 BIT(9)
> > +
> > +#endif
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-08-29 18:32 ` Laurent Pinchart
@ 2020-08-31 7:38 ` Jacopo Mondi
2020-08-31 14:17 ` Laurent Pinchart
0 siblings, 1 reply; 111+ messages in thread
From: Jacopo Mondi @ 2020-08-31 7:38 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, Kieran Bingham, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> Hi Jacopo,
>
> On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> > Hi Laurent, Dave, Naush,
> >
> > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > From: Naushir Patuck <naush@raspberrypi.com>
> > >
> > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > Compared to the bcm2835-camera driver present in staging, this driver
> > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > the VC4 firmware running on the VPU.
> > >
> > > The commit is made up of a series of changes cherry-picked from the
> > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > additional enhancements, forward-ported to the mainline kernel.
> > >
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > ---
> > > Changes since v1:
> > >
> > > - Re-fetch mbus code from subdev on a g_fmt call
> > > - Group all ioctl disabling together
> > > - Fix reference counting in unicam_open
> > > - Add support for VIDIOC_[S|G]_SELECTION
> > > ---
> >
> > [snip]
> >
> > > +
> > > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > > +{
> > > + struct platform_device *pdev = dev->pdev;
> > > + struct v4l2_fwnode_endpoint ep = { 0 };
> > > + struct device_node *ep_node;
> > > + struct device_node *sensor_node;
> > > + unsigned int lane;
> > > + int ret = -EINVAL;
> > > +
> > > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > > + &dev->max_data_lanes) < 0) {
> > > + unicam_err(dev, "number of data lanes not set\n");
> > > + return -EINVAL;
> > > + }
> >
> > mmm, this seems like a good use case for the newly introduced
> > get_mbus_config pad operation, isn't it ?
> >
> > We could drop the custom "brcm,num-data-lanes" property, use the
> > standard "num-lanes" to describe the number of wired data lanes, and
> > get the number of actually used ones at s_stream(1) time with the new pad
> > operation
> >
> > What do you think ? This would require implemeting the operation in
> > each subdev the UNICAM interface has to work with, but this doesn't
> > sound too much of a work.
> >
> > For reference this is how it happens on rcar+adv748x:
> > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
>
> Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> The former tells how many data lanes the unicam IP has (the same IP,
> with the same compatible string, is instantiated in different SoCs with
> different number of lanes), while the latter queries at runtime how many
> data lanes the remote subdev will use for its current configuration. The
Can't we get the number of available data lanes just by parsing the
endpoint ? I'm aware of the implication that the 'data-lanes' property
would differ in the bridge and sensor endpoints, but now that we have
a mechanism to get that information wihtout parsing the remote
endpoint it doesn't seem wrong to me.
And that's not different from what it's done in R-Car (half-true: we
have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
as you said the number of active data lanes can be downscaled to
transmit low resolution streams).
I think Dave already had this conversation in the past, I might dig
the archive and search for a reference.
> DT property serves as a sanity check, to ensure to no more than the
> number of supported data lanes is used, while the subdev operation
> should be used in the unicam driver to support sources that will
> transmit over a different number of data lanes depending on the
> configuration (mostly depending on the resolution). We don't have to
I think reading the remote's mbus configuration might also help in
support connecting pluggable devices with a different number of wired
data lanes than the bridge's available ones.
Am I missing something maybe? The non-connected lanes on the bride
will simply stay unused, can't they ?
> implement support for get_mbus_config just yet, it can be delayed until
> we have a source that requires it.
I sketched a few patches yesterday basically appling what's done in
5cef0785362ed and a61b1b5d41735. Is a custom property better in your
opinion ?
Thanks
j
>
> > > +
> > > + /* Get the local endpoint and remote device. */
> > > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > > + if (!ep_node) {
> > > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > > + return -EINVAL;
> > > + }
> > > +
> > > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > > +
> > > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > > + if (!sensor_node) {
> > > + unicam_dbg(3, dev, "can't get remote parent\n");
> > > + goto cleanup_exit;
> > > + }
> > > +
> > > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > > +
> > > + /* Parse the local endpoint and validate its configuration. */
> > > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > > +
> > > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > > + ep.bus_type);
> > > +
> > > + dev->bus_type = ep.bus_type;
> > > +
> > > + switch (ep.bus_type) {
> > > + case V4L2_MBUS_CSI2_DPHY:
> > > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > > + case 1:
> > > + case 2:
> > > + case 4:
> > > + break;
> > > +
> > > + default:
> > > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > > + sensor_node,
> > > + ep.bus.mipi_csi2.num_data_lanes);
> > > + goto cleanup_exit;
> > > + }
> > > +
> > > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > > + sensor_node);
> > > + goto cleanup_exit;
> > > + }
> > > + }
> > > +
> > > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > > + ep.bus.mipi_csi2.num_data_lanes,
> > > + dev->max_data_lanes);
> > > + }
> > > +
> > > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > > +
> > > + break;
> > > +
> > > + case V4L2_MBUS_CCP2:
> > > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > > + ep.bus.mipi_csi1.data_lane != 1) {
> > > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > > + sensor_node);
> > > + goto cleanup_exit;
> > > + }
> > > +
> > > + dev->max_data_lanes = 1;
> > > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > > + break;
> > > +
> > > + default:
> > > + /* Unsupported bus type */
> > > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > > + sensor_node, ep.bus_type);
> > > + goto cleanup_exit;
> > > + }
> > > +
> > > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > > + sensor_node,
> > > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > > + dev->max_data_lanes, dev->bus_flags);
> > > +
> > > + /* Initialize and register the async notifier. */
> > > + v4l2_async_notifier_init(&dev->notifier);
> > > + dev->notifier.ops = &unicam_async_ops;
> > > +
> > > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > > + if (ret) {
> > > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > > + goto cleanup_exit;
> > > + }
> > > +
> > > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > > + if (ret) {
> > > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > > + ret = -EINVAL;
> > > + }
> > > +
> > > +cleanup_exit:
> > > + of_node_put(sensor_node);
> > > + of_node_put(ep_node);
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +static int unicam_probe(struct platform_device *pdev)
> > > +{
> > > + struct unicam_device *unicam;
> > > + int ret;
> > > +
> > > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > > + if (!unicam)
> > > + return -ENOMEM;
> > > +
> > > + kref_init(&unicam->kref);
> > > + unicam->pdev = pdev;
> > > +
> > > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > > + if (IS_ERR(unicam->base)) {
> > > + unicam_err(unicam, "Failed to get main io block\n");
> > > + ret = PTR_ERR(unicam->base);
> > > + goto err_unicam_put;
> > > + }
> > > +
> > > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > > + if (IS_ERR(unicam->clk_gate_base)) {
> > > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > > + ret = PTR_ERR(unicam->clk_gate_base);
> > > + goto err_unicam_put;
> > > + }
> > > +
> > > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > > + if (IS_ERR(unicam->clock)) {
> > > + unicam_err(unicam, "Failed to get clock\n");
> > > + ret = PTR_ERR(unicam->clock);
> > > + goto err_unicam_put;
> > > + }
> > > +
> > > + ret = platform_get_irq(pdev, 0);
> > > + if (ret <= 0) {
> > > + dev_err(&pdev->dev, "No IRQ resource\n");
> > > + ret = -EINVAL;
> > > + goto err_unicam_put;
> > > + }
> > > +
> > > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > > + "unicam_capture0", unicam);
> > > + if (ret) {
> > > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > > + ret = -EINVAL;
> > > + goto err_unicam_put;
> > > + }
> > > +
> > > + unicam->mdev.dev = &pdev->dev;
> > > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > + sizeof(unicam->mdev.model));
> > > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > > + "platform:%s", dev_name(&pdev->dev));
> > > + unicam->mdev.hw_revision = 0;
> > > +
> > > + media_device_init(&unicam->mdev);
> > > +
> > > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > > +
> > > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > > + if (ret) {
> > > + unicam_err(unicam,
> > > + "Unable to register v4l2 device.\n");
> > > + goto err_unicam_put;
> > > + }
> > > +
> > > + ret = media_device_register(&unicam->mdev);
> > > + if (ret < 0) {
> > > + unicam_err(unicam,
> > > + "Unable to register media-controller device.\n");
> > > + goto err_v4l2_unregister;
> > > + }
> > > +
> > > + /* Reserve space for the controls */
> > > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > > + if (ret < 0)
> > > + goto err_media_unregister;
> > > +
> > > + /* set the driver data in platform device */
> > > + platform_set_drvdata(pdev, unicam);
> > > +
> > > + ret = of_unicam_connect_subdevs(unicam);
> > > + if (ret) {
> > > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > > + goto err_media_unregister;
> > > + }
> > > +
> > > + /* Enable the block power domain */
> > > + pm_runtime_enable(&pdev->dev);
> > > +
> > > + return 0;
> > > +
> > > +err_media_unregister:
> > > + media_device_unregister(&unicam->mdev);
> > > +err_v4l2_unregister:
> > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > +err_unicam_put:
> > > + unicam_put(unicam);
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +static int unicam_remove(struct platform_device *pdev)
> > > +{
> > > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > > +
> > > + unicam_dbg(2, unicam, "%s\n", __func__);
> > > +
> > > + v4l2_async_notifier_unregister(&unicam->notifier);
> > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > + media_device_unregister(&unicam->mdev);
> > > + unregister_nodes(unicam);
> > > +
> > > + pm_runtime_disable(&pdev->dev);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static const struct of_device_id unicam_of_match[] = {
> > > + { .compatible = "brcm,bcm2835-unicam", },
> > > + { /* sentinel */ },
> > > +};
> > > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > > +
> > > +static struct platform_driver unicam_driver = {
> > > + .probe = unicam_probe,
> > > + .remove = unicam_remove,
> > > + .driver = {
> > > + .name = UNICAM_MODULE_NAME,
> > > + .of_match_table = of_match_ptr(unicam_of_match),
> > > + },
> > > +};
> > > +
> > > +module_platform_driver(unicam_driver);
> > > +
> > > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > > +MODULE_LICENSE("GPL");
> > > +MODULE_VERSION(UNICAM_VERSION);
> > > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > new file mode 100644
> > > index 000000000000..ae059a171d0f
> > > --- /dev/null
> > > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > @@ -0,0 +1,253 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > +
> > > +/*
> > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > + */
> > > +
> > > +#ifndef VC4_REGS_UNICAM_H
> > > +#define VC4_REGS_UNICAM_H
> > > +
> > > +/*
> > > + * The following values are taken from files found within the code drop
> > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > + * They have been modified to be only the register offset.
> > > + */
> > > +#define UNICAM_CTRL 0x000
> > > +#define UNICAM_STA 0x004
> > > +#define UNICAM_ANA 0x008
> > > +#define UNICAM_PRI 0x00c
> > > +#define UNICAM_CLK 0x010
> > > +#define UNICAM_CLT 0x014
> > > +#define UNICAM_DAT0 0x018
> > > +#define UNICAM_DAT1 0x01c
> > > +#define UNICAM_DAT2 0x020
> > > +#define UNICAM_DAT3 0x024
> > > +#define UNICAM_DLT 0x028
> > > +#define UNICAM_CMP0 0x02c
> > > +#define UNICAM_CMP1 0x030
> > > +#define UNICAM_CAP0 0x034
> > > +#define UNICAM_CAP1 0x038
> > > +#define UNICAM_ICTL 0x100
> > > +#define UNICAM_ISTA 0x104
> > > +#define UNICAM_IDI0 0x108
> > > +#define UNICAM_IPIPE 0x10c
> > > +#define UNICAM_IBSA0 0x110
> > > +#define UNICAM_IBEA0 0x114
> > > +#define UNICAM_IBLS 0x118
> > > +#define UNICAM_IBWP 0x11c
> > > +#define UNICAM_IHWIN 0x120
> > > +#define UNICAM_IHSTA 0x124
> > > +#define UNICAM_IVWIN 0x128
> > > +#define UNICAM_IVSTA 0x12c
> > > +#define UNICAM_ICC 0x130
> > > +#define UNICAM_ICS 0x134
> > > +#define UNICAM_IDC 0x138
> > > +#define UNICAM_IDPO 0x13c
> > > +#define UNICAM_IDCA 0x140
> > > +#define UNICAM_IDCD 0x144
> > > +#define UNICAM_IDS 0x148
> > > +#define UNICAM_DCS 0x200
> > > +#define UNICAM_DBSA0 0x204
> > > +#define UNICAM_DBEA0 0x208
> > > +#define UNICAM_DBWP 0x20c
> > > +#define UNICAM_DBCTL 0x300
> > > +#define UNICAM_IBSA1 0x304
> > > +#define UNICAM_IBEA1 0x308
> > > +#define UNICAM_IDI1 0x30c
> > > +#define UNICAM_DBSA1 0x310
> > > +#define UNICAM_DBEA1 0x314
> > > +#define UNICAM_MISC 0x400
> > > +
> > > +/*
> > > + * The following bitmasks are from the kernel released by Broadcom
> > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > + * Unicam block as BCM2835, as defined in eg
> > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > + *
> > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > + */
> > > +/* UNICAM_CTRL Register */
> > > +#define UNICAM_CPE BIT(0)
> > > +#define UNICAM_MEM BIT(1)
> > > +#define UNICAM_CPR BIT(2)
> > > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > > +#define UNICAM_CPM_CSI2 0
> > > +#define UNICAM_CPM_CCP2 1
> > > +#define UNICAM_SOE BIT(4)
> > > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > > +#define UNICAM_DCM_STROBE 0
> > > +#define UNICAM_DCM_DATA 1
> > > +#define UNICAM_SLS BIT(6)
> > > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > > +
> > > +/* UNICAM_STA Register */
> > > +#define UNICAM_SYN BIT(0)
> > > +#define UNICAM_CS BIT(1)
> > > +#define UNICAM_SBE BIT(2)
> > > +#define UNICAM_PBE BIT(3)
> > > +#define UNICAM_HOE BIT(4)
> > > +#define UNICAM_PLE BIT(5)
> > > +#define UNICAM_SSC BIT(6)
> > > +#define UNICAM_CRCE BIT(7)
> > > +#define UNICAM_OES BIT(8)
> > > +#define UNICAM_IFO BIT(9)
> > > +#define UNICAM_OFO BIT(10)
> > > +#define UNICAM_BFO BIT(11)
> > > +#define UNICAM_DL BIT(12)
> > > +#define UNICAM_PS BIT(13)
> > > +#define UNICAM_IS BIT(14)
> > > +#define UNICAM_PI0 BIT(15)
> > > +#define UNICAM_PI1 BIT(16)
> > > +#define UNICAM_FSI_S BIT(17)
> > > +#define UNICAM_FEI_S BIT(18)
> > > +#define UNICAM_LCI_S BIT(19)
> > > +#define UNICAM_BUF0_RDY BIT(20)
> > > +#define UNICAM_BUF0_NO BIT(21)
> > > +#define UNICAM_BUF1_RDY BIT(22)
> > > +#define UNICAM_BUF1_NO BIT(23)
> > > +#define UNICAM_DI BIT(24)
> > > +
> > > +#define UNICAM_STA_MASK_ALL \
> > > + (UNICAM_DL + \
> > > + UNICAM_SBE + \
> > > + UNICAM_PBE + \
> > > + UNICAM_HOE + \
> > > + UNICAM_PLE + \
> > > + UNICAM_SSC + \
> > > + UNICAM_CRCE + \
> > > + UNICAM_IFO + \
> > > + UNICAM_OFO + \
> > > + UNICAM_PS + \
> > > + UNICAM_PI0 + \
> > > + UNICAM_PI1)
> > > +
> > > +/* UNICAM_ANA Register */
> > > +#define UNICAM_APD BIT(0)
> > > +#define UNICAM_BPD BIT(1)
> > > +#define UNICAM_AR BIT(2)
> > > +#define UNICAM_DDL BIT(3)
> > > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > > +
> > > +/* UNICAM_PRI Register */
> > > +#define UNICAM_PE BIT(0)
> > > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > > +
> > > +/* UNICAM_CLK Register */
> > > +#define UNICAM_CLE BIT(0)
> > > +#define UNICAM_CLPD BIT(1)
> > > +#define UNICAM_CLLPE BIT(2)
> > > +#define UNICAM_CLHSE BIT(3)
> > > +#define UNICAM_CLTRE BIT(4)
> > > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > > +#define UNICAM_CLSTE BIT(29)
> > > +
> > > +/* UNICAM_CLT Register */
> > > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > > +
> > > +/* UNICAM_DATn Registers */
> > > +#define UNICAM_DLE BIT(0)
> > > +#define UNICAM_DLPD BIT(1)
> > > +#define UNICAM_DLLPE BIT(2)
> > > +#define UNICAM_DLHSE BIT(3)
> > > +#define UNICAM_DLTRE BIT(4)
> > > +#define UNICAM_DLSM BIT(5)
> > > +#define UNICAM_DLFO BIT(28)
> > > +#define UNICAM_DLSTE BIT(29)
> > > +
> > > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > > +
> > > +/* UNICAM_DLT Register */
> > > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > > +
> > > +/* UNICAM_ICTL Register */
> > > +#define UNICAM_FSIE BIT(0)
> > > +#define UNICAM_FEIE BIT(1)
> > > +#define UNICAM_IBOB BIT(2)
> > > +#define UNICAM_FCM BIT(3)
> > > +#define UNICAM_TFC BIT(4)
> > > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > > +
> > > +/* UNICAM_IDI0/1 Register */
> > > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > > +
> > > +/* UNICAM_ISTA Register */
> > > +#define UNICAM_FSI BIT(0)
> > > +#define UNICAM_FEI BIT(1)
> > > +#define UNICAM_LCI BIT(2)
> > > +
> > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > > +
> > > +/* UNICAM_IPIPE Register */
> > > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > > + /* Unpacking modes */
> > > + #define UNICAM_PUM_NONE 0
> > > + #define UNICAM_PUM_UNPACK6 1
> > > + #define UNICAM_PUM_UNPACK7 2
> > > + #define UNICAM_PUM_UNPACK8 3
> > > + #define UNICAM_PUM_UNPACK10 4
> > > + #define UNICAM_PUM_UNPACK12 5
> > > + #define UNICAM_PUM_UNPACK14 6
> > > + #define UNICAM_PUM_UNPACK16 7
> > > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > > + /* Packing modes */
> > > + #define UNICAM_PPM_NONE 0
> > > + #define UNICAM_PPM_PACK8 1
> > > + #define UNICAM_PPM_PACK10 2
> > > + #define UNICAM_PPM_PACK12 3
> > > + #define UNICAM_PPM_PACK14 4
> > > + #define UNICAM_PPM_PACK16 5
> > > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > > +
> > > +/* UNICAM_ICC Register */
> > > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > > +
> > > +/* UNICAM_DCS Register */
> > > +#define UNICAM_DIE BIT(0)
> > > +#define UNICAM_DIM BIT(1)
> > > +#define UNICAM_DBOB BIT(3)
> > > +#define UNICAM_FDE BIT(4)
> > > +#define UNICAM_LDP BIT(5)
> > > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > > +
> > > +/* UNICAM_DBCTL Register */
> > > +#define UNICAM_DBEN BIT(0)
> > > +#define UNICAM_BUF0_IE BIT(1)
> > > +#define UNICAM_BUF1_IE BIT(2)
> > > +
> > > +/* UNICAM_CMP[0,1] register */
> > > +#define UNICAM_PCE BIT(31)
> > > +#define UNICAM_GI BIT(9)
> > > +#define UNICAM_CPH BIT(8)
> > > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > > +
> > > +/* UNICAM_MISC register */
> > > +#define UNICAM_FL0 BIT(6)
> > > +#define UNICAM_FL1 BIT(9)
> > > +
> > > +#endif
>
> --
> Regards,
>
> Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-08-31 7:38 ` Jacopo Mondi
@ 2020-08-31 14:17 ` Laurent Pinchart
2020-08-31 14:46 ` Jacopo Mondi
0 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-08-31 14:17 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, Kieran Bingham, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Jacopo,
On Mon, Aug 31, 2020 at 09:38:58AM +0200, Jacopo Mondi wrote:
> On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> > On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> > > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > > From: Naushir Patuck <naush@raspberrypi.com>
> > > >
> > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > Compared to the bcm2835-camera driver present in staging, this driver
> > > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > > the VC4 firmware running on the VPU.
> > > >
> > > > The commit is made up of a series of changes cherry-picked from the
> > > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > > additional enhancements, forward-ported to the mainline kernel.
> > > >
> > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > ---
> > > > Changes since v1:
> > > >
> > > > - Re-fetch mbus code from subdev on a g_fmt call
> > > > - Group all ioctl disabling together
> > > > - Fix reference counting in unicam_open
> > > > - Add support for VIDIOC_[S|G]_SELECTION
> > > > ---
> > >
> > > [snip]
> > >
> > > > +
> > > > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > > > +{
> > > > + struct platform_device *pdev = dev->pdev;
> > > > + struct v4l2_fwnode_endpoint ep = { 0 };
> > > > + struct device_node *ep_node;
> > > > + struct device_node *sensor_node;
> > > > + unsigned int lane;
> > > > + int ret = -EINVAL;
> > > > +
> > > > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > > > + &dev->max_data_lanes) < 0) {
> > > > + unicam_err(dev, "number of data lanes not set\n");
> > > > + return -EINVAL;
> > > > + }
> > >
> > > mmm, this seems like a good use case for the newly introduced
> > > get_mbus_config pad operation, isn't it ?
> > >
> > > We could drop the custom "brcm,num-data-lanes" property, use the
> > > standard "num-lanes" to describe the number of wired data lanes, and
> > > get the number of actually used ones at s_stream(1) time with the new pad
> > > operation
> > >
> > > What do you think ? This would require implemeting the operation in
> > > each subdev the UNICAM interface has to work with, but this doesn't
> > > sound too much of a work.
> > >
> > > For reference this is how it happens on rcar+adv748x:
> > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
> >
> > Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> > The former tells how many data lanes the unicam IP has (the same IP,
> > with the same compatible string, is instantiated in different SoCs with
> > different number of lanes), while the latter queries at runtime how many
> > data lanes the remote subdev will use for its current configuration. The
>
> Can't we get the number of available data lanes just by parsing the
> endpoint ?
We can get the number of data lanes routed on the board, which may not
match the number of data lanes available in the IP. I'll let Dave
comment on this, I recall we had a lengthy discussion on this topic in
the past, but I don't remember the details :-S
> I'm aware of the implication that the 'data-lanes' property
> would differ in the bridge and sensor endpoints, but now that we have
> a mechanism to get that information wihtout parsing the remote
> endpoint it doesn't seem wrong to me.
That's not the issue at hand. The data-lanes property need to match on
both sides of a link, as it tells how many data lanes are routed on the
board. Of course that becomes problematic when we start using overlays
for pluggable boards, as the two sides can be designed with a different
number of routed lanes.
> And that's not different from what it's done in R-Car (half-true: we
> have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
> as you said the number of active data lanes can be downscaled to
> transmit low resolution streams).
>
> I think Dave already had this conversation in the past, I might dig
> the archive and search for a reference.
>
> > DT property serves as a sanity check, to ensure to no more than the
> > number of supported data lanes is used, while the subdev operation
> > should be used in the unicam driver to support sources that will
> > transmit over a different number of data lanes depending on the
> > configuration (mostly depending on the resolution). We don't have to
>
> I think reading the remote's mbus configuration might also help in
> support connecting pluggable devices with a different number of wired
> data lanes than the bridge's available ones.
No, that we shouldn't do. DT properties need to be interpreted within
the context of a compatible string, reading properties of a remote node
is a no-go in most cases. It's really really something that must not be
done without careful consideration, and is never a simple way to work
around a problem.
> Am I missing something maybe? The non-connected lanes on the bride
> will simply stay unused, can't they ?
>
> > implement support for get_mbus_config just yet, it can be delayed until
> > we have a source that requires it.
>
> I sketched a few patches yesterday basically appling what's done in
> 5cef0785362ed and a61b1b5d41735. Is a custom property better in your
> opinion ?
Again, the custom property only specifies the number of data lanes
implemented in the unicam IP core. It's nothing else than that, and does
*not* convey any information about how many data lanes the sensor will
use.
> > > > +
> > > > + /* Get the local endpoint and remote device. */
> > > > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > > > + if (!ep_node) {
> > > > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > > > + return -EINVAL;
> > > > + }
> > > > +
> > > > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > > > +
> > > > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > > > + if (!sensor_node) {
> > > > + unicam_dbg(3, dev, "can't get remote parent\n");
> > > > + goto cleanup_exit;
> > > > + }
> > > > +
> > > > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > > > +
> > > > + /* Parse the local endpoint and validate its configuration. */
> > > > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > > > +
> > > > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > > > + ep.bus_type);
> > > > +
> > > > + dev->bus_type = ep.bus_type;
> > > > +
> > > > + switch (ep.bus_type) {
> > > > + case V4L2_MBUS_CSI2_DPHY:
> > > > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > > > + case 1:
> > > > + case 2:
> > > > + case 4:
> > > > + break;
> > > > +
> > > > + default:
> > > > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > > > + sensor_node,
> > > > + ep.bus.mipi_csi2.num_data_lanes);
> > > > + goto cleanup_exit;
> > > > + }
> > > > +
> > > > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > > > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > > > + sensor_node);
> > > > + goto cleanup_exit;
> > > > + }
> > > > + }
> > > > +
> > > > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > > > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > > > + ep.bus.mipi_csi2.num_data_lanes,
> > > > + dev->max_data_lanes);
> > > > + }
> > > > +
> > > > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > > > +
> > > > + break;
> > > > +
> > > > + case V4L2_MBUS_CCP2:
> > > > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > > > + ep.bus.mipi_csi1.data_lane != 1) {
> > > > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > > > + sensor_node);
> > > > + goto cleanup_exit;
> > > > + }
> > > > +
> > > > + dev->max_data_lanes = 1;
> > > > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > > > + break;
> > > > +
> > > > + default:
> > > > + /* Unsupported bus type */
> > > > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > > > + sensor_node, ep.bus_type);
> > > > + goto cleanup_exit;
> > > > + }
> > > > +
> > > > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > > > + sensor_node,
> > > > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > > > + dev->max_data_lanes, dev->bus_flags);
> > > > +
> > > > + /* Initialize and register the async notifier. */
> > > > + v4l2_async_notifier_init(&dev->notifier);
> > > > + dev->notifier.ops = &unicam_async_ops;
> > > > +
> > > > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > > > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > > > + if (ret) {
> > > > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > > > + goto cleanup_exit;
> > > > + }
> > > > +
> > > > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > > > + if (ret) {
> > > > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > > > + ret = -EINVAL;
> > > > + }
> > > > +
> > > > +cleanup_exit:
> > > > + of_node_put(sensor_node);
> > > > + of_node_put(ep_node);
> > > > +
> > > > + return ret;
> > > > +}
> > > > +
> > > > +static int unicam_probe(struct platform_device *pdev)
> > > > +{
> > > > + struct unicam_device *unicam;
> > > > + int ret;
> > > > +
> > > > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > > > + if (!unicam)
> > > > + return -ENOMEM;
> > > > +
> > > > + kref_init(&unicam->kref);
> > > > + unicam->pdev = pdev;
> > > > +
> > > > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > > > + if (IS_ERR(unicam->base)) {
> > > > + unicam_err(unicam, "Failed to get main io block\n");
> > > > + ret = PTR_ERR(unicam->base);
> > > > + goto err_unicam_put;
> > > > + }
> > > > +
> > > > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > > > + if (IS_ERR(unicam->clk_gate_base)) {
> > > > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > > > + ret = PTR_ERR(unicam->clk_gate_base);
> > > > + goto err_unicam_put;
> > > > + }
> > > > +
> > > > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > > > + if (IS_ERR(unicam->clock)) {
> > > > + unicam_err(unicam, "Failed to get clock\n");
> > > > + ret = PTR_ERR(unicam->clock);
> > > > + goto err_unicam_put;
> > > > + }
> > > > +
> > > > + ret = platform_get_irq(pdev, 0);
> > > > + if (ret <= 0) {
> > > > + dev_err(&pdev->dev, "No IRQ resource\n");
> > > > + ret = -EINVAL;
> > > > + goto err_unicam_put;
> > > > + }
> > > > +
> > > > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > > > + "unicam_capture0", unicam);
> > > > + if (ret) {
> > > > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > > > + ret = -EINVAL;
> > > > + goto err_unicam_put;
> > > > + }
> > > > +
> > > > + unicam->mdev.dev = &pdev->dev;
> > > > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > > + sizeof(unicam->mdev.model));
> > > > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > > > + "platform:%s", dev_name(&pdev->dev));
> > > > + unicam->mdev.hw_revision = 0;
> > > > +
> > > > + media_device_init(&unicam->mdev);
> > > > +
> > > > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > > > +
> > > > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > > > + if (ret) {
> > > > + unicam_err(unicam,
> > > > + "Unable to register v4l2 device.\n");
> > > > + goto err_unicam_put;
> > > > + }
> > > > +
> > > > + ret = media_device_register(&unicam->mdev);
> > > > + if (ret < 0) {
> > > > + unicam_err(unicam,
> > > > + "Unable to register media-controller device.\n");
> > > > + goto err_v4l2_unregister;
> > > > + }
> > > > +
> > > > + /* Reserve space for the controls */
> > > > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > > > + if (ret < 0)
> > > > + goto err_media_unregister;
> > > > +
> > > > + /* set the driver data in platform device */
> > > > + platform_set_drvdata(pdev, unicam);
> > > > +
> > > > + ret = of_unicam_connect_subdevs(unicam);
> > > > + if (ret) {
> > > > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > > > + goto err_media_unregister;
> > > > + }
> > > > +
> > > > + /* Enable the block power domain */
> > > > + pm_runtime_enable(&pdev->dev);
> > > > +
> > > > + return 0;
> > > > +
> > > > +err_media_unregister:
> > > > + media_device_unregister(&unicam->mdev);
> > > > +err_v4l2_unregister:
> > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > +err_unicam_put:
> > > > + unicam_put(unicam);
> > > > +
> > > > + return ret;
> > > > +}
> > > > +
> > > > +static int unicam_remove(struct platform_device *pdev)
> > > > +{
> > > > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > > > +
> > > > + unicam_dbg(2, unicam, "%s\n", __func__);
> > > > +
> > > > + v4l2_async_notifier_unregister(&unicam->notifier);
> > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > + media_device_unregister(&unicam->mdev);
> > > > + unregister_nodes(unicam);
> > > > +
> > > > + pm_runtime_disable(&pdev->dev);
> > > > +
> > > > + return 0;
> > > > +}
> > > > +
> > > > +static const struct of_device_id unicam_of_match[] = {
> > > > + { .compatible = "brcm,bcm2835-unicam", },
> > > > + { /* sentinel */ },
> > > > +};
> > > > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > > > +
> > > > +static struct platform_driver unicam_driver = {
> > > > + .probe = unicam_probe,
> > > > + .remove = unicam_remove,
> > > > + .driver = {
> > > > + .name = UNICAM_MODULE_NAME,
> > > > + .of_match_table = of_match_ptr(unicam_of_match),
> > > > + },
> > > > +};
> > > > +
> > > > +module_platform_driver(unicam_driver);
> > > > +
> > > > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > > > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > > > +MODULE_LICENSE("GPL");
> > > > +MODULE_VERSION(UNICAM_VERSION);
> > > > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > new file mode 100644
> > > > index 000000000000..ae059a171d0f
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > @@ -0,0 +1,253 @@
> > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > +
> > > > +/*
> > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > + */
> > > > +
> > > > +#ifndef VC4_REGS_UNICAM_H
> > > > +#define VC4_REGS_UNICAM_H
> > > > +
> > > > +/*
> > > > + * The following values are taken from files found within the code drop
> > > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > > + * They have been modified to be only the register offset.
> > > > + */
> > > > +#define UNICAM_CTRL 0x000
> > > > +#define UNICAM_STA 0x004
> > > > +#define UNICAM_ANA 0x008
> > > > +#define UNICAM_PRI 0x00c
> > > > +#define UNICAM_CLK 0x010
> > > > +#define UNICAM_CLT 0x014
> > > > +#define UNICAM_DAT0 0x018
> > > > +#define UNICAM_DAT1 0x01c
> > > > +#define UNICAM_DAT2 0x020
> > > > +#define UNICAM_DAT3 0x024
> > > > +#define UNICAM_DLT 0x028
> > > > +#define UNICAM_CMP0 0x02c
> > > > +#define UNICAM_CMP1 0x030
> > > > +#define UNICAM_CAP0 0x034
> > > > +#define UNICAM_CAP1 0x038
> > > > +#define UNICAM_ICTL 0x100
> > > > +#define UNICAM_ISTA 0x104
> > > > +#define UNICAM_IDI0 0x108
> > > > +#define UNICAM_IPIPE 0x10c
> > > > +#define UNICAM_IBSA0 0x110
> > > > +#define UNICAM_IBEA0 0x114
> > > > +#define UNICAM_IBLS 0x118
> > > > +#define UNICAM_IBWP 0x11c
> > > > +#define UNICAM_IHWIN 0x120
> > > > +#define UNICAM_IHSTA 0x124
> > > > +#define UNICAM_IVWIN 0x128
> > > > +#define UNICAM_IVSTA 0x12c
> > > > +#define UNICAM_ICC 0x130
> > > > +#define UNICAM_ICS 0x134
> > > > +#define UNICAM_IDC 0x138
> > > > +#define UNICAM_IDPO 0x13c
> > > > +#define UNICAM_IDCA 0x140
> > > > +#define UNICAM_IDCD 0x144
> > > > +#define UNICAM_IDS 0x148
> > > > +#define UNICAM_DCS 0x200
> > > > +#define UNICAM_DBSA0 0x204
> > > > +#define UNICAM_DBEA0 0x208
> > > > +#define UNICAM_DBWP 0x20c
> > > > +#define UNICAM_DBCTL 0x300
> > > > +#define UNICAM_IBSA1 0x304
> > > > +#define UNICAM_IBEA1 0x308
> > > > +#define UNICAM_IDI1 0x30c
> > > > +#define UNICAM_DBSA1 0x310
> > > > +#define UNICAM_DBEA1 0x314
> > > > +#define UNICAM_MISC 0x400
> > > > +
> > > > +/*
> > > > + * The following bitmasks are from the kernel released by Broadcom
> > > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > > + * Unicam block as BCM2835, as defined in eg
> > > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > > + *
> > > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > > + */
> > > > +/* UNICAM_CTRL Register */
> > > > +#define UNICAM_CPE BIT(0)
> > > > +#define UNICAM_MEM BIT(1)
> > > > +#define UNICAM_CPR BIT(2)
> > > > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > > > +#define UNICAM_CPM_CSI2 0
> > > > +#define UNICAM_CPM_CCP2 1
> > > > +#define UNICAM_SOE BIT(4)
> > > > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > > > +#define UNICAM_DCM_STROBE 0
> > > > +#define UNICAM_DCM_DATA 1
> > > > +#define UNICAM_SLS BIT(6)
> > > > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > > > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > > > +
> > > > +/* UNICAM_STA Register */
> > > > +#define UNICAM_SYN BIT(0)
> > > > +#define UNICAM_CS BIT(1)
> > > > +#define UNICAM_SBE BIT(2)
> > > > +#define UNICAM_PBE BIT(3)
> > > > +#define UNICAM_HOE BIT(4)
> > > > +#define UNICAM_PLE BIT(5)
> > > > +#define UNICAM_SSC BIT(6)
> > > > +#define UNICAM_CRCE BIT(7)
> > > > +#define UNICAM_OES BIT(8)
> > > > +#define UNICAM_IFO BIT(9)
> > > > +#define UNICAM_OFO BIT(10)
> > > > +#define UNICAM_BFO BIT(11)
> > > > +#define UNICAM_DL BIT(12)
> > > > +#define UNICAM_PS BIT(13)
> > > > +#define UNICAM_IS BIT(14)
> > > > +#define UNICAM_PI0 BIT(15)
> > > > +#define UNICAM_PI1 BIT(16)
> > > > +#define UNICAM_FSI_S BIT(17)
> > > > +#define UNICAM_FEI_S BIT(18)
> > > > +#define UNICAM_LCI_S BIT(19)
> > > > +#define UNICAM_BUF0_RDY BIT(20)
> > > > +#define UNICAM_BUF0_NO BIT(21)
> > > > +#define UNICAM_BUF1_RDY BIT(22)
> > > > +#define UNICAM_BUF1_NO BIT(23)
> > > > +#define UNICAM_DI BIT(24)
> > > > +
> > > > +#define UNICAM_STA_MASK_ALL \
> > > > + (UNICAM_DL + \
> > > > + UNICAM_SBE + \
> > > > + UNICAM_PBE + \
> > > > + UNICAM_HOE + \
> > > > + UNICAM_PLE + \
> > > > + UNICAM_SSC + \
> > > > + UNICAM_CRCE + \
> > > > + UNICAM_IFO + \
> > > > + UNICAM_OFO + \
> > > > + UNICAM_PS + \
> > > > + UNICAM_PI0 + \
> > > > + UNICAM_PI1)
> > > > +
> > > > +/* UNICAM_ANA Register */
> > > > +#define UNICAM_APD BIT(0)
> > > > +#define UNICAM_BPD BIT(1)
> > > > +#define UNICAM_AR BIT(2)
> > > > +#define UNICAM_DDL BIT(3)
> > > > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > > > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > > > +
> > > > +/* UNICAM_PRI Register */
> > > > +#define UNICAM_PE BIT(0)
> > > > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > > > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > > > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > > > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > > > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > > > +
> > > > +/* UNICAM_CLK Register */
> > > > +#define UNICAM_CLE BIT(0)
> > > > +#define UNICAM_CLPD BIT(1)
> > > > +#define UNICAM_CLLPE BIT(2)
> > > > +#define UNICAM_CLHSE BIT(3)
> > > > +#define UNICAM_CLTRE BIT(4)
> > > > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > > > +#define UNICAM_CLSTE BIT(29)
> > > > +
> > > > +/* UNICAM_CLT Register */
> > > > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > > > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > > > +
> > > > +/* UNICAM_DATn Registers */
> > > > +#define UNICAM_DLE BIT(0)
> > > > +#define UNICAM_DLPD BIT(1)
> > > > +#define UNICAM_DLLPE BIT(2)
> > > > +#define UNICAM_DLHSE BIT(3)
> > > > +#define UNICAM_DLTRE BIT(4)
> > > > +#define UNICAM_DLSM BIT(5)
> > > > +#define UNICAM_DLFO BIT(28)
> > > > +#define UNICAM_DLSTE BIT(29)
> > > > +
> > > > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > > > +
> > > > +/* UNICAM_DLT Register */
> > > > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > > > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > > > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > > > +
> > > > +/* UNICAM_ICTL Register */
> > > > +#define UNICAM_FSIE BIT(0)
> > > > +#define UNICAM_FEIE BIT(1)
> > > > +#define UNICAM_IBOB BIT(2)
> > > > +#define UNICAM_FCM BIT(3)
> > > > +#define UNICAM_TFC BIT(4)
> > > > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > > > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > > > +
> > > > +/* UNICAM_IDI0/1 Register */
> > > > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > > > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > > > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > > > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > > > +
> > > > +/* UNICAM_ISTA Register */
> > > > +#define UNICAM_FSI BIT(0)
> > > > +#define UNICAM_FEI BIT(1)
> > > > +#define UNICAM_LCI BIT(2)
> > > > +
> > > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > > > +
> > > > +/* UNICAM_IPIPE Register */
> > > > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > > > + /* Unpacking modes */
> > > > + #define UNICAM_PUM_NONE 0
> > > > + #define UNICAM_PUM_UNPACK6 1
> > > > + #define UNICAM_PUM_UNPACK7 2
> > > > + #define UNICAM_PUM_UNPACK8 3
> > > > + #define UNICAM_PUM_UNPACK10 4
> > > > + #define UNICAM_PUM_UNPACK12 5
> > > > + #define UNICAM_PUM_UNPACK14 6
> > > > + #define UNICAM_PUM_UNPACK16 7
> > > > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > > > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > > > + /* Packing modes */
> > > > + #define UNICAM_PPM_NONE 0
> > > > + #define UNICAM_PPM_PACK8 1
> > > > + #define UNICAM_PPM_PACK10 2
> > > > + #define UNICAM_PPM_PACK12 3
> > > > + #define UNICAM_PPM_PACK14 4
> > > > + #define UNICAM_PPM_PACK16 5
> > > > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > > > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > > > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > > > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > > > +
> > > > +/* UNICAM_ICC Register */
> > > > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > > > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > > > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > > > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > > > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > > > +
> > > > +/* UNICAM_DCS Register */
> > > > +#define UNICAM_DIE BIT(0)
> > > > +#define UNICAM_DIM BIT(1)
> > > > +#define UNICAM_DBOB BIT(3)
> > > > +#define UNICAM_FDE BIT(4)
> > > > +#define UNICAM_LDP BIT(5)
> > > > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > > > +
> > > > +/* UNICAM_DBCTL Register */
> > > > +#define UNICAM_DBEN BIT(0)
> > > > +#define UNICAM_BUF0_IE BIT(1)
> > > > +#define UNICAM_BUF1_IE BIT(2)
> > > > +
> > > > +/* UNICAM_CMP[0,1] register */
> > > > +#define UNICAM_PCE BIT(31)
> > > > +#define UNICAM_GI BIT(9)
> > > > +#define UNICAM_CPH BIT(8)
> > > > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > > > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > > > +
> > > > +/* UNICAM_MISC register */
> > > > +#define UNICAM_FL0 BIT(6)
> > > > +#define UNICAM_FL1 BIT(9)
> > > > +
> > > > +#endif
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-08-31 14:17 ` Laurent Pinchart
@ 2020-08-31 14:46 ` Jacopo Mondi
2020-08-31 14:56 ` Laurent Pinchart
0 siblings, 1 reply; 111+ messages in thread
From: Jacopo Mondi @ 2020-08-31 14:46 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, Kieran Bingham, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Laurent,
I'll let David reply but
On Mon, Aug 31, 2020 at 05:17:04PM +0300, Laurent Pinchart wrote:
> Hi Jacopo,
>
> On Mon, Aug 31, 2020 at 09:38:58AM +0200, Jacopo Mondi wrote:
> > On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> > > On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> > > > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > > > From: Naushir Patuck <naush@raspberrypi.com>
> > > > >
> > > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > > Compared to the bcm2835-camera driver present in staging, this driver
> > > > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > > > the VC4 firmware running on the VPU.
> > > > >
> > > > > The commit is made up of a series of changes cherry-picked from the
> > > > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > > > additional enhancements, forward-ported to the mainline kernel.
> > > > >
> > > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > > ---
> > > > > Changes since v1:
> > > > >
> > > > > - Re-fetch mbus code from subdev on a g_fmt call
> > > > > - Group all ioctl disabling together
> > > > > - Fix reference counting in unicam_open
> > > > > - Add support for VIDIOC_[S|G]_SELECTION
> > > > > ---
> > > >
> > > > [snip]
> > > >
> > > > > +
> > > > > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > > > > +{
> > > > > + struct platform_device *pdev = dev->pdev;
> > > > > + struct v4l2_fwnode_endpoint ep = { 0 };
> > > > > + struct device_node *ep_node;
> > > > > + struct device_node *sensor_node;
> > > > > + unsigned int lane;
> > > > > + int ret = -EINVAL;
> > > > > +
> > > > > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > > > > + &dev->max_data_lanes) < 0) {
> > > > > + unicam_err(dev, "number of data lanes not set\n");
> > > > > + return -EINVAL;
> > > > > + }
> > > >
> > > > mmm, this seems like a good use case for the newly introduced
> > > > get_mbus_config pad operation, isn't it ?
> > > >
> > > > We could drop the custom "brcm,num-data-lanes" property, use the
> > > > standard "num-lanes" to describe the number of wired data lanes, and
> > > > get the number of actually used ones at s_stream(1) time with the new pad
> > > > operation
> > > >
> > > > What do you think ? This would require implemeting the operation in
> > > > each subdev the UNICAM interface has to work with, but this doesn't
> > > > sound too much of a work.
> > > >
> > > > For reference this is how it happens on rcar+adv748x:
> > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
> > >
> > > Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> > > The former tells how many data lanes the unicam IP has (the same IP,
> > > with the same compatible string, is instantiated in different SoCs with
> > > different number of lanes), while the latter queries at runtime how many
> > > data lanes the remote subdev will use for its current configuration. The
> >
> > Can't we get the number of available data lanes just by parsing the
> > endpoint ?
>
> We can get the number of data lanes routed on the board, which may not
> match the number of data lanes available in the IP. I'll let Dave
> comment on this, I recall we had a lengthy discussion on this topic in
> the past, but I don't remember the details :-S
>
I think the max number of number of lanes in the IP should come in DT
by restricting the number of accepted data-lanes value depending on
the compatible string.
> > I'm aware of the implication that the 'data-lanes' property
> > would differ in the bridge and sensor endpoints, but now that we have
> > a mechanism to get that information wihtout parsing the remote
> > endpoint it doesn't seem wrong to me.
>
> That's not the issue at hand. The data-lanes property need to match on
> both sides of a link, as it tells how many data lanes are routed on the
> board. Of course that becomes problematic when we start using overlays
> for pluggable boards, as the two sides can be designed with a different
> number of routed lanes.
>
It actually is. As I read the current implementation, UNICAM has 4
lanes available, 2 are enabled in the endpoint, and the remote uses 2.
If we set the number of lanes in the endpoint to 4 (the actually
available ones) and use only the two required by querying the remote
end at s_stream time, we don't need any custom sourcery
> > And that's not different from what it's done in R-Car (half-true: we
> > have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
> > as you said the number of active data lanes can be downscaled to
> > transmit low resolution streams).
> >
> > I think Dave already had this conversation in the past, I might dig
> > the archive and search for a reference.
> >
> > > DT property serves as a sanity check, to ensure to no more than the
> > > number of supported data lanes is used, while the subdev operation
> > > should be used in the unicam driver to support sources that will
> > > transmit over a different number of data lanes depending on the
> > > configuration (mostly depending on the resolution). We don't have to
> >
> > I think reading the remote's mbus configuration might also help in
> > support connecting pluggable devices with a different number of wired
> > data lanes than the bridge's available ones.
>
> No, that we shouldn't do. DT properties need to be interpreted within
> the context of a compatible string, reading properties of a remote node
> is a no-go in most cases. It's really really something that must not be
> done without careful consideration, and is never a simple way to work
> around a problem.
What I mean is that get_mbus_config() allows you to retrieve those
information -without- having to read the remote endpoint, which has
always been a blocker for this kind of operations (and I agree it
shouldn't be done).
>
> > Am I missing something maybe? The non-connected lanes on the bride
> > will simply stay unused, can't they ?
> >
> > > implement support for get_mbus_config just yet, it can be delayed until
> > > we have a source that requires it.
> >
> > I sketched a few patches yesterday basically appling what's done in
> > 5cef0785362ed and a61b1b5d41735. Is a custom property better in your
> > opinion ?
>
> Again, the custom property only specifies the number of data lanes
> implemented in the unicam IP core. It's nothing else than that, and does
> *not* convey any information about how many data lanes the sensor will
> use.
>
It shouldn't, and to me that information should come from DT as we do
for all other platforms.
Let's see, if feels a bit a waste to use a custom property for
something we can do without, and usage of get_mbus_config() would
allow to support switching devices at run-time with a different number
of data-lanes, as use case I know Dave has downstream.
Thanks
j
> > > > > +
> > > > > + /* Get the local endpoint and remote device. */
> > > > > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > > > > + if (!ep_node) {
> > > > > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > > > > + return -EINVAL;
> > > > > + }
> > > > > +
> > > > > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > > > > +
> > > > > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > > > > + if (!sensor_node) {
> > > > > + unicam_dbg(3, dev, "can't get remote parent\n");
> > > > > + goto cleanup_exit;
> > > > > + }
> > > > > +
> > > > > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > > > > +
> > > > > + /* Parse the local endpoint and validate its configuration. */
> > > > > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > > > > +
> > > > > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > > > > + ep.bus_type);
> > > > > +
> > > > > + dev->bus_type = ep.bus_type;
> > > > > +
> > > > > + switch (ep.bus_type) {
> > > > > + case V4L2_MBUS_CSI2_DPHY:
> > > > > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > > > > + case 1:
> > > > > + case 2:
> > > > > + case 4:
> > > > > + break;
> > > > > +
> > > > > + default:
> > > > > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > > > > + sensor_node,
> > > > > + ep.bus.mipi_csi2.num_data_lanes);
> > > > > + goto cleanup_exit;
> > > > > + }
> > > > > +
> > > > > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > > > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > > > > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > > > > + sensor_node);
> > > > > + goto cleanup_exit;
> > > > > + }
> > > > > + }
> > > > > +
> > > > > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > > > > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > > > > + ep.bus.mipi_csi2.num_data_lanes,
> > > > > + dev->max_data_lanes);
> > > > > + }
> > > > > +
> > > > > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > > > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > > > > +
> > > > > + break;
> > > > > +
> > > > > + case V4L2_MBUS_CCP2:
> > > > > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > > > > + ep.bus.mipi_csi1.data_lane != 1) {
> > > > > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > > > > + sensor_node);
> > > > > + goto cleanup_exit;
> > > > > + }
> > > > > +
> > > > > + dev->max_data_lanes = 1;
> > > > > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > > > > + break;
> > > > > +
> > > > > + default:
> > > > > + /* Unsupported bus type */
> > > > > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > > > > + sensor_node, ep.bus_type);
> > > > > + goto cleanup_exit;
> > > > > + }
> > > > > +
> > > > > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > > > > + sensor_node,
> > > > > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > > > > + dev->max_data_lanes, dev->bus_flags);
> > > > > +
> > > > > + /* Initialize and register the async notifier. */
> > > > > + v4l2_async_notifier_init(&dev->notifier);
> > > > > + dev->notifier.ops = &unicam_async_ops;
> > > > > +
> > > > > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > > > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > > > > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > > > > + if (ret) {
> > > > > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > > > > + goto cleanup_exit;
> > > > > + }
> > > > > +
> > > > > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > > > > + if (ret) {
> > > > > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > > > > + ret = -EINVAL;
> > > > > + }
> > > > > +
> > > > > +cleanup_exit:
> > > > > + of_node_put(sensor_node);
> > > > > + of_node_put(ep_node);
> > > > > +
> > > > > + return ret;
> > > > > +}
> > > > > +
> > > > > +static int unicam_probe(struct platform_device *pdev)
> > > > > +{
> > > > > + struct unicam_device *unicam;
> > > > > + int ret;
> > > > > +
> > > > > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > > > > + if (!unicam)
> > > > > + return -ENOMEM;
> > > > > +
> > > > > + kref_init(&unicam->kref);
> > > > > + unicam->pdev = pdev;
> > > > > +
> > > > > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > > > > + if (IS_ERR(unicam->base)) {
> > > > > + unicam_err(unicam, "Failed to get main io block\n");
> > > > > + ret = PTR_ERR(unicam->base);
> > > > > + goto err_unicam_put;
> > > > > + }
> > > > > +
> > > > > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > > > > + if (IS_ERR(unicam->clk_gate_base)) {
> > > > > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > > > > + ret = PTR_ERR(unicam->clk_gate_base);
> > > > > + goto err_unicam_put;
> > > > > + }
> > > > > +
> > > > > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > > > > + if (IS_ERR(unicam->clock)) {
> > > > > + unicam_err(unicam, "Failed to get clock\n");
> > > > > + ret = PTR_ERR(unicam->clock);
> > > > > + goto err_unicam_put;
> > > > > + }
> > > > > +
> > > > > + ret = platform_get_irq(pdev, 0);
> > > > > + if (ret <= 0) {
> > > > > + dev_err(&pdev->dev, "No IRQ resource\n");
> > > > > + ret = -EINVAL;
> > > > > + goto err_unicam_put;
> > > > > + }
> > > > > +
> > > > > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > > > > + "unicam_capture0", unicam);
> > > > > + if (ret) {
> > > > > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > > > > + ret = -EINVAL;
> > > > > + goto err_unicam_put;
> > > > > + }
> > > > > +
> > > > > + unicam->mdev.dev = &pdev->dev;
> > > > > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > > > + sizeof(unicam->mdev.model));
> > > > > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > > > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > > > > + "platform:%s", dev_name(&pdev->dev));
> > > > > + unicam->mdev.hw_revision = 0;
> > > > > +
> > > > > + media_device_init(&unicam->mdev);
> > > > > +
> > > > > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > > > > +
> > > > > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > > > > + if (ret) {
> > > > > + unicam_err(unicam,
> > > > > + "Unable to register v4l2 device.\n");
> > > > > + goto err_unicam_put;
> > > > > + }
> > > > > +
> > > > > + ret = media_device_register(&unicam->mdev);
> > > > > + if (ret < 0) {
> > > > > + unicam_err(unicam,
> > > > > + "Unable to register media-controller device.\n");
> > > > > + goto err_v4l2_unregister;
> > > > > + }
> > > > > +
> > > > > + /* Reserve space for the controls */
> > > > > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > > > > + if (ret < 0)
> > > > > + goto err_media_unregister;
> > > > > +
> > > > > + /* set the driver data in platform device */
> > > > > + platform_set_drvdata(pdev, unicam);
> > > > > +
> > > > > + ret = of_unicam_connect_subdevs(unicam);
> > > > > + if (ret) {
> > > > > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > > > > + goto err_media_unregister;
> > > > > + }
> > > > > +
> > > > > + /* Enable the block power domain */
> > > > > + pm_runtime_enable(&pdev->dev);
> > > > > +
> > > > > + return 0;
> > > > > +
> > > > > +err_media_unregister:
> > > > > + media_device_unregister(&unicam->mdev);
> > > > > +err_v4l2_unregister:
> > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > +err_unicam_put:
> > > > > + unicam_put(unicam);
> > > > > +
> > > > > + return ret;
> > > > > +}
> > > > > +
> > > > > +static int unicam_remove(struct platform_device *pdev)
> > > > > +{
> > > > > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > > > > +
> > > > > + unicam_dbg(2, unicam, "%s\n", __func__);
> > > > > +
> > > > > + v4l2_async_notifier_unregister(&unicam->notifier);
> > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > + media_device_unregister(&unicam->mdev);
> > > > > + unregister_nodes(unicam);
> > > > > +
> > > > > + pm_runtime_disable(&pdev->dev);
> > > > > +
> > > > > + return 0;
> > > > > +}
> > > > > +
> > > > > +static const struct of_device_id unicam_of_match[] = {
> > > > > + { .compatible = "brcm,bcm2835-unicam", },
> > > > > + { /* sentinel */ },
> > > > > +};
> > > > > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > > > > +
> > > > > +static struct platform_driver unicam_driver = {
> > > > > + .probe = unicam_probe,
> > > > > + .remove = unicam_remove,
> > > > > + .driver = {
> > > > > + .name = UNICAM_MODULE_NAME,
> > > > > + .of_match_table = of_match_ptr(unicam_of_match),
> > > > > + },
> > > > > +};
> > > > > +
> > > > > +module_platform_driver(unicam_driver);
> > > > > +
> > > > > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > > > > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > > > > +MODULE_LICENSE("GPL");
> > > > > +MODULE_VERSION(UNICAM_VERSION);
> > > > > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > new file mode 100644
> > > > > index 000000000000..ae059a171d0f
> > > > > --- /dev/null
> > > > > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > @@ -0,0 +1,253 @@
> > > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > > +
> > > > > +/*
> > > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > + */
> > > > > +
> > > > > +#ifndef VC4_REGS_UNICAM_H
> > > > > +#define VC4_REGS_UNICAM_H
> > > > > +
> > > > > +/*
> > > > > + * The following values are taken from files found within the code drop
> > > > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > > > + * They have been modified to be only the register offset.
> > > > > + */
> > > > > +#define UNICAM_CTRL 0x000
> > > > > +#define UNICAM_STA 0x004
> > > > > +#define UNICAM_ANA 0x008
> > > > > +#define UNICAM_PRI 0x00c
> > > > > +#define UNICAM_CLK 0x010
> > > > > +#define UNICAM_CLT 0x014
> > > > > +#define UNICAM_DAT0 0x018
> > > > > +#define UNICAM_DAT1 0x01c
> > > > > +#define UNICAM_DAT2 0x020
> > > > > +#define UNICAM_DAT3 0x024
> > > > > +#define UNICAM_DLT 0x028
> > > > > +#define UNICAM_CMP0 0x02c
> > > > > +#define UNICAM_CMP1 0x030
> > > > > +#define UNICAM_CAP0 0x034
> > > > > +#define UNICAM_CAP1 0x038
> > > > > +#define UNICAM_ICTL 0x100
> > > > > +#define UNICAM_ISTA 0x104
> > > > > +#define UNICAM_IDI0 0x108
> > > > > +#define UNICAM_IPIPE 0x10c
> > > > > +#define UNICAM_IBSA0 0x110
> > > > > +#define UNICAM_IBEA0 0x114
> > > > > +#define UNICAM_IBLS 0x118
> > > > > +#define UNICAM_IBWP 0x11c
> > > > > +#define UNICAM_IHWIN 0x120
> > > > > +#define UNICAM_IHSTA 0x124
> > > > > +#define UNICAM_IVWIN 0x128
> > > > > +#define UNICAM_IVSTA 0x12c
> > > > > +#define UNICAM_ICC 0x130
> > > > > +#define UNICAM_ICS 0x134
> > > > > +#define UNICAM_IDC 0x138
> > > > > +#define UNICAM_IDPO 0x13c
> > > > > +#define UNICAM_IDCA 0x140
> > > > > +#define UNICAM_IDCD 0x144
> > > > > +#define UNICAM_IDS 0x148
> > > > > +#define UNICAM_DCS 0x200
> > > > > +#define UNICAM_DBSA0 0x204
> > > > > +#define UNICAM_DBEA0 0x208
> > > > > +#define UNICAM_DBWP 0x20c
> > > > > +#define UNICAM_DBCTL 0x300
> > > > > +#define UNICAM_IBSA1 0x304
> > > > > +#define UNICAM_IBEA1 0x308
> > > > > +#define UNICAM_IDI1 0x30c
> > > > > +#define UNICAM_DBSA1 0x310
> > > > > +#define UNICAM_DBEA1 0x314
> > > > > +#define UNICAM_MISC 0x400
> > > > > +
> > > > > +/*
> > > > > + * The following bitmasks are from the kernel released by Broadcom
> > > > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > > > + * Unicam block as BCM2835, as defined in eg
> > > > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > > > + *
> > > > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > > > + */
> > > > > +/* UNICAM_CTRL Register */
> > > > > +#define UNICAM_CPE BIT(0)
> > > > > +#define UNICAM_MEM BIT(1)
> > > > > +#define UNICAM_CPR BIT(2)
> > > > > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > > > > +#define UNICAM_CPM_CSI2 0
> > > > > +#define UNICAM_CPM_CCP2 1
> > > > > +#define UNICAM_SOE BIT(4)
> > > > > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > > > > +#define UNICAM_DCM_STROBE 0
> > > > > +#define UNICAM_DCM_DATA 1
> > > > > +#define UNICAM_SLS BIT(6)
> > > > > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > > > > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > > > > +
> > > > > +/* UNICAM_STA Register */
> > > > > +#define UNICAM_SYN BIT(0)
> > > > > +#define UNICAM_CS BIT(1)
> > > > > +#define UNICAM_SBE BIT(2)
> > > > > +#define UNICAM_PBE BIT(3)
> > > > > +#define UNICAM_HOE BIT(4)
> > > > > +#define UNICAM_PLE BIT(5)
> > > > > +#define UNICAM_SSC BIT(6)
> > > > > +#define UNICAM_CRCE BIT(7)
> > > > > +#define UNICAM_OES BIT(8)
> > > > > +#define UNICAM_IFO BIT(9)
> > > > > +#define UNICAM_OFO BIT(10)
> > > > > +#define UNICAM_BFO BIT(11)
> > > > > +#define UNICAM_DL BIT(12)
> > > > > +#define UNICAM_PS BIT(13)
> > > > > +#define UNICAM_IS BIT(14)
> > > > > +#define UNICAM_PI0 BIT(15)
> > > > > +#define UNICAM_PI1 BIT(16)
> > > > > +#define UNICAM_FSI_S BIT(17)
> > > > > +#define UNICAM_FEI_S BIT(18)
> > > > > +#define UNICAM_LCI_S BIT(19)
> > > > > +#define UNICAM_BUF0_RDY BIT(20)
> > > > > +#define UNICAM_BUF0_NO BIT(21)
> > > > > +#define UNICAM_BUF1_RDY BIT(22)
> > > > > +#define UNICAM_BUF1_NO BIT(23)
> > > > > +#define UNICAM_DI BIT(24)
> > > > > +
> > > > > +#define UNICAM_STA_MASK_ALL \
> > > > > + (UNICAM_DL + \
> > > > > + UNICAM_SBE + \
> > > > > + UNICAM_PBE + \
> > > > > + UNICAM_HOE + \
> > > > > + UNICAM_PLE + \
> > > > > + UNICAM_SSC + \
> > > > > + UNICAM_CRCE + \
> > > > > + UNICAM_IFO + \
> > > > > + UNICAM_OFO + \
> > > > > + UNICAM_PS + \
> > > > > + UNICAM_PI0 + \
> > > > > + UNICAM_PI1)
> > > > > +
> > > > > +/* UNICAM_ANA Register */
> > > > > +#define UNICAM_APD BIT(0)
> > > > > +#define UNICAM_BPD BIT(1)
> > > > > +#define UNICAM_AR BIT(2)
> > > > > +#define UNICAM_DDL BIT(3)
> > > > > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > > > > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > > > > +
> > > > > +/* UNICAM_PRI Register */
> > > > > +#define UNICAM_PE BIT(0)
> > > > > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > > > > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > > > > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > > > > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > > > > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > > > > +
> > > > > +/* UNICAM_CLK Register */
> > > > > +#define UNICAM_CLE BIT(0)
> > > > > +#define UNICAM_CLPD BIT(1)
> > > > > +#define UNICAM_CLLPE BIT(2)
> > > > > +#define UNICAM_CLHSE BIT(3)
> > > > > +#define UNICAM_CLTRE BIT(4)
> > > > > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > > > > +#define UNICAM_CLSTE BIT(29)
> > > > > +
> > > > > +/* UNICAM_CLT Register */
> > > > > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > > > > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > > > > +
> > > > > +/* UNICAM_DATn Registers */
> > > > > +#define UNICAM_DLE BIT(0)
> > > > > +#define UNICAM_DLPD BIT(1)
> > > > > +#define UNICAM_DLLPE BIT(2)
> > > > > +#define UNICAM_DLHSE BIT(3)
> > > > > +#define UNICAM_DLTRE BIT(4)
> > > > > +#define UNICAM_DLSM BIT(5)
> > > > > +#define UNICAM_DLFO BIT(28)
> > > > > +#define UNICAM_DLSTE BIT(29)
> > > > > +
> > > > > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > > > > +
> > > > > +/* UNICAM_DLT Register */
> > > > > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > > > > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > > > > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > > > > +
> > > > > +/* UNICAM_ICTL Register */
> > > > > +#define UNICAM_FSIE BIT(0)
> > > > > +#define UNICAM_FEIE BIT(1)
> > > > > +#define UNICAM_IBOB BIT(2)
> > > > > +#define UNICAM_FCM BIT(3)
> > > > > +#define UNICAM_TFC BIT(4)
> > > > > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > > > > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > > > > +
> > > > > +/* UNICAM_IDI0/1 Register */
> > > > > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > > > > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > > > > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > > > > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > > > > +
> > > > > +/* UNICAM_ISTA Register */
> > > > > +#define UNICAM_FSI BIT(0)
> > > > > +#define UNICAM_FEI BIT(1)
> > > > > +#define UNICAM_LCI BIT(2)
> > > > > +
> > > > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > > > > +
> > > > > +/* UNICAM_IPIPE Register */
> > > > > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > > > > + /* Unpacking modes */
> > > > > + #define UNICAM_PUM_NONE 0
> > > > > + #define UNICAM_PUM_UNPACK6 1
> > > > > + #define UNICAM_PUM_UNPACK7 2
> > > > > + #define UNICAM_PUM_UNPACK8 3
> > > > > + #define UNICAM_PUM_UNPACK10 4
> > > > > + #define UNICAM_PUM_UNPACK12 5
> > > > > + #define UNICAM_PUM_UNPACK14 6
> > > > > + #define UNICAM_PUM_UNPACK16 7
> > > > > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > > > > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > > > > + /* Packing modes */
> > > > > + #define UNICAM_PPM_NONE 0
> > > > > + #define UNICAM_PPM_PACK8 1
> > > > > + #define UNICAM_PPM_PACK10 2
> > > > > + #define UNICAM_PPM_PACK12 3
> > > > > + #define UNICAM_PPM_PACK14 4
> > > > > + #define UNICAM_PPM_PACK16 5
> > > > > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > > > > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > > > > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > > > > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > > > > +
> > > > > +/* UNICAM_ICC Register */
> > > > > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > > > > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > > > > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > > > > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > > > > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > > > > +
> > > > > +/* UNICAM_DCS Register */
> > > > > +#define UNICAM_DIE BIT(0)
> > > > > +#define UNICAM_DIM BIT(1)
> > > > > +#define UNICAM_DBOB BIT(3)
> > > > > +#define UNICAM_FDE BIT(4)
> > > > > +#define UNICAM_LDP BIT(5)
> > > > > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > > > > +
> > > > > +/* UNICAM_DBCTL Register */
> > > > > +#define UNICAM_DBEN BIT(0)
> > > > > +#define UNICAM_BUF0_IE BIT(1)
> > > > > +#define UNICAM_BUF1_IE BIT(2)
> > > > > +
> > > > > +/* UNICAM_CMP[0,1] register */
> > > > > +#define UNICAM_PCE BIT(31)
> > > > > +#define UNICAM_GI BIT(9)
> > > > > +#define UNICAM_CPH BIT(8)
> > > > > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > > > > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > > > > +
> > > > > +/* UNICAM_MISC register */
> > > > > +#define UNICAM_FL0 BIT(6)
> > > > > +#define UNICAM_FL1 BIT(9)
> > > > > +
> > > > > +#endif
>
> --
> Regards,
>
> Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-08-31 14:46 ` Jacopo Mondi
@ 2020-08-31 14:56 ` Laurent Pinchart
2020-09-01 8:41 ` Dave Stevenson
0 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-08-31 14:56 UTC (permalink / raw)
To: Jacopo Mondi
Cc: linux-media, Kieran Bingham, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Jacopo,
On Mon, Aug 31, 2020 at 04:46:37PM +0200, Jacopo Mondi wrote:
> On Mon, Aug 31, 2020 at 05:17:04PM +0300, Laurent Pinchart wrote:
> > On Mon, Aug 31, 2020 at 09:38:58AM +0200, Jacopo Mondi wrote:
> > > On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> > > > On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> > > > > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > > > > From: Naushir Patuck <naush@raspberrypi.com>
> > > > > >
> > > > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > > > Compared to the bcm2835-camera driver present in staging, this driver
> > > > > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > > > > the VC4 firmware running on the VPU.
> > > > > >
> > > > > > The commit is made up of a series of changes cherry-picked from the
> > > > > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > > > > additional enhancements, forward-ported to the mainline kernel.
> > > > > >
> > > > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > > > ---
> > > > > > Changes since v1:
> > > > > >
> > > > > > - Re-fetch mbus code from subdev on a g_fmt call
> > > > > > - Group all ioctl disabling together
> > > > > > - Fix reference counting in unicam_open
> > > > > > - Add support for VIDIOC_[S|G]_SELECTION
> > > > > > ---
> > > > >
> > > > > [snip]
> > > > >
> > > > > > +
> > > > > > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > > > > > +{
> > > > > > + struct platform_device *pdev = dev->pdev;
> > > > > > + struct v4l2_fwnode_endpoint ep = { 0 };
> > > > > > + struct device_node *ep_node;
> > > > > > + struct device_node *sensor_node;
> > > > > > + unsigned int lane;
> > > > > > + int ret = -EINVAL;
> > > > > > +
> > > > > > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > > > > > + &dev->max_data_lanes) < 0) {
> > > > > > + unicam_err(dev, "number of data lanes not set\n");
> > > > > > + return -EINVAL;
> > > > > > + }
> > > > >
> > > > > mmm, this seems like a good use case for the newly introduced
> > > > > get_mbus_config pad operation, isn't it ?
> > > > >
> > > > > We could drop the custom "brcm,num-data-lanes" property, use the
> > > > > standard "num-lanes" to describe the number of wired data lanes, and
> > > > > get the number of actually used ones at s_stream(1) time with the new pad
> > > > > operation
> > > > >
> > > > > What do you think ? This would require implemeting the operation in
> > > > > each subdev the UNICAM interface has to work with, but this doesn't
> > > > > sound too much of a work.
> > > > >
> > > > > For reference this is how it happens on rcar+adv748x:
> > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
> > > >
> > > > Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> > > > The former tells how many data lanes the unicam IP has (the same IP,
> > > > with the same compatible string, is instantiated in different SoCs with
> > > > different number of lanes), while the latter queries at runtime how many
> > > > data lanes the remote subdev will use for its current configuration. The
> > >
> > > Can't we get the number of available data lanes just by parsing the
> > > endpoint ?
> >
> > We can get the number of data lanes routed on the board, which may not
> > match the number of data lanes available in the IP. I'll let Dave
> > comment on this, I recall we had a lengthy discussion on this topic in
> > the past, but I don't remember the details :-S
>
> I think the max number of number of lanes in the IP should come in DT
> by restricting the number of accepted data-lanes value depending on
> the compatible string.
Unless I'm mistaken we have a single compatibility string.
> > > I'm aware of the implication that the 'data-lanes' property
> > > would differ in the bridge and sensor endpoints, but now that we have
> > > a mechanism to get that information wihtout parsing the remote
> > > endpoint it doesn't seem wrong to me.
> >
> > That's not the issue at hand. The data-lanes property need to match on
> > both sides of a link, as it tells how many data lanes are routed on the
> > board. Of course that becomes problematic when we start using overlays
> > for pluggable boards, as the two sides can be designed with a different
> > number of routed lanes.
>
> It actually is. As I read the current implementation, UNICAM has 4
> lanes available, 2 are enabled in the endpoint, and the remote uses 2.
That's because the board has 2 data lanes routed.
> If we set the number of lanes in the endpoint to 4 (the actually
> available ones) and use only the two required by querying the remote
> end at s_stream time, we don't need any custom sourcery
That would be incorrect, as there are two data lanes routed, so you
shouldn't set the data-lanes property to 4 lanes in the endpoint.
> > > And that's not different from what it's done in R-Car (half-true: we
> > > have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
> > > as you said the number of active data lanes can be downscaled to
> > > transmit low resolution streams).
> > >
> > > I think Dave already had this conversation in the past, I might dig
> > > the archive and search for a reference.
> > >
> > > > DT property serves as a sanity check, to ensure to no more than the
> > > > number of supported data lanes is used, while the subdev operation
> > > > should be used in the unicam driver to support sources that will
> > > > transmit over a different number of data lanes depending on the
> > > > configuration (mostly depending on the resolution). We don't have to
> > >
> > > I think reading the remote's mbus configuration might also help in
> > > support connecting pluggable devices with a different number of wired
> > > data lanes than the bridge's available ones.
> >
> > No, that we shouldn't do. DT properties need to be interpreted within
> > the context of a compatible string, reading properties of a remote node
> > is a no-go in most cases. It's really really something that must not be
> > done without careful consideration, and is never a simple way to work
> > around a problem.
>
> What I mean is that get_mbus_config() allows you to retrieve those
> information -without- having to read the remote endpoint, which has
> always been a blocker for this kind of operations (and I agree it
> shouldn't be done).
>
> > > Am I missing something maybe? The non-connected lanes on the bride
> > > will simply stay unused, can't they ?
> > >
> > > > implement support for get_mbus_config just yet, it can be delayed until
> > > > we have a source that requires it.
> > >
> > > I sketched a few patches yesterday basically appling what's done in
> > > 5cef0785362ed and a61b1b5d41735. Is a custom property better in your
> > > opinion ?
> >
> > Again, the custom property only specifies the number of data lanes
> > implemented in the unicam IP core. It's nothing else than that, and does
> > *not* convey any information about how many data lanes the sensor will
> > use.
>
> It shouldn't, and to me that information should come from DT as we do
> for all other platforms.
>
> Let's see, if feels a bit a waste to use a custom property for
> something we can do without, and usage of get_mbus_config() would
> allow to support switching devices at run-time with a different number
> of data-lanes, as use case I know Dave has downstream.
This is *still* not what brcm,num-lanes is about... get_mbus_config()
will not tell you how many data lanes the unicam IP supports.
> > > > > > +
> > > > > > + /* Get the local endpoint and remote device. */
> > > > > > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > > > > > + if (!ep_node) {
> > > > > > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > > > > > + return -EINVAL;
> > > > > > + }
> > > > > > +
> > > > > > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > > > > > +
> > > > > > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > > > > > + if (!sensor_node) {
> > > > > > + unicam_dbg(3, dev, "can't get remote parent\n");
> > > > > > + goto cleanup_exit;
> > > > > > + }
> > > > > > +
> > > > > > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > > > > > +
> > > > > > + /* Parse the local endpoint and validate its configuration. */
> > > > > > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > > > > > +
> > > > > > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > > > > > + ep.bus_type);
> > > > > > +
> > > > > > + dev->bus_type = ep.bus_type;
> > > > > > +
> > > > > > + switch (ep.bus_type) {
> > > > > > + case V4L2_MBUS_CSI2_DPHY:
> > > > > > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > > > > > + case 1:
> > > > > > + case 2:
> > > > > > + case 4:
> > > > > > + break;
> > > > > > +
> > > > > > + default:
> > > > > > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > > > > > + sensor_node,
> > > > > > + ep.bus.mipi_csi2.num_data_lanes);
> > > > > > + goto cleanup_exit;
> > > > > > + }
> > > > > > +
> > > > > > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > > > > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > > > > > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > > > > > + sensor_node);
> > > > > > + goto cleanup_exit;
> > > > > > + }
> > > > > > + }
> > > > > > +
> > > > > > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > > > > > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > > > > > + ep.bus.mipi_csi2.num_data_lanes,
> > > > > > + dev->max_data_lanes);
> > > > > > + }
> > > > > > +
> > > > > > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > > > > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > > > > > +
> > > > > > + break;
> > > > > > +
> > > > > > + case V4L2_MBUS_CCP2:
> > > > > > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > > > > > + ep.bus.mipi_csi1.data_lane != 1) {
> > > > > > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > > > > > + sensor_node);
> > > > > > + goto cleanup_exit;
> > > > > > + }
> > > > > > +
> > > > > > + dev->max_data_lanes = 1;
> > > > > > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > > > > > + break;
> > > > > > +
> > > > > > + default:
> > > > > > + /* Unsupported bus type */
> > > > > > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > > > > > + sensor_node, ep.bus_type);
> > > > > > + goto cleanup_exit;
> > > > > > + }
> > > > > > +
> > > > > > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > > > > > + sensor_node,
> > > > > > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > > > > > + dev->max_data_lanes, dev->bus_flags);
> > > > > > +
> > > > > > + /* Initialize and register the async notifier. */
> > > > > > + v4l2_async_notifier_init(&dev->notifier);
> > > > > > + dev->notifier.ops = &unicam_async_ops;
> > > > > > +
> > > > > > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > > > > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > > > > > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > > > > > + if (ret) {
> > > > > > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > > > > > + goto cleanup_exit;
> > > > > > + }
> > > > > > +
> > > > > > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > > > > > + if (ret) {
> > > > > > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > > > > > + ret = -EINVAL;
> > > > > > + }
> > > > > > +
> > > > > > +cleanup_exit:
> > > > > > + of_node_put(sensor_node);
> > > > > > + of_node_put(ep_node);
> > > > > > +
> > > > > > + return ret;
> > > > > > +}
> > > > > > +
> > > > > > +static int unicam_probe(struct platform_device *pdev)
> > > > > > +{
> > > > > > + struct unicam_device *unicam;
> > > > > > + int ret;
> > > > > > +
> > > > > > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > > > > > + if (!unicam)
> > > > > > + return -ENOMEM;
> > > > > > +
> > > > > > + kref_init(&unicam->kref);
> > > > > > + unicam->pdev = pdev;
> > > > > > +
> > > > > > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > > > > > + if (IS_ERR(unicam->base)) {
> > > > > > + unicam_err(unicam, "Failed to get main io block\n");
> > > > > > + ret = PTR_ERR(unicam->base);
> > > > > > + goto err_unicam_put;
> > > > > > + }
> > > > > > +
> > > > > > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > > > > > + if (IS_ERR(unicam->clk_gate_base)) {
> > > > > > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > > > > > + ret = PTR_ERR(unicam->clk_gate_base);
> > > > > > + goto err_unicam_put;
> > > > > > + }
> > > > > > +
> > > > > > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > > > > > + if (IS_ERR(unicam->clock)) {
> > > > > > + unicam_err(unicam, "Failed to get clock\n");
> > > > > > + ret = PTR_ERR(unicam->clock);
> > > > > > + goto err_unicam_put;
> > > > > > + }
> > > > > > +
> > > > > > + ret = platform_get_irq(pdev, 0);
> > > > > > + if (ret <= 0) {
> > > > > > + dev_err(&pdev->dev, "No IRQ resource\n");
> > > > > > + ret = -EINVAL;
> > > > > > + goto err_unicam_put;
> > > > > > + }
> > > > > > +
> > > > > > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > > > > > + "unicam_capture0", unicam);
> > > > > > + if (ret) {
> > > > > > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > > > > > + ret = -EINVAL;
> > > > > > + goto err_unicam_put;
> > > > > > + }
> > > > > > +
> > > > > > + unicam->mdev.dev = &pdev->dev;
> > > > > > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > > > > + sizeof(unicam->mdev.model));
> > > > > > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > > > > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > > > > > + "platform:%s", dev_name(&pdev->dev));
> > > > > > + unicam->mdev.hw_revision = 0;
> > > > > > +
> > > > > > + media_device_init(&unicam->mdev);
> > > > > > +
> > > > > > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > > > > > +
> > > > > > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > > > > > + if (ret) {
> > > > > > + unicam_err(unicam,
> > > > > > + "Unable to register v4l2 device.\n");
> > > > > > + goto err_unicam_put;
> > > > > > + }
> > > > > > +
> > > > > > + ret = media_device_register(&unicam->mdev);
> > > > > > + if (ret < 0) {
> > > > > > + unicam_err(unicam,
> > > > > > + "Unable to register media-controller device.\n");
> > > > > > + goto err_v4l2_unregister;
> > > > > > + }
> > > > > > +
> > > > > > + /* Reserve space for the controls */
> > > > > > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > > > > > + if (ret < 0)
> > > > > > + goto err_media_unregister;
> > > > > > +
> > > > > > + /* set the driver data in platform device */
> > > > > > + platform_set_drvdata(pdev, unicam);
> > > > > > +
> > > > > > + ret = of_unicam_connect_subdevs(unicam);
> > > > > > + if (ret) {
> > > > > > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > > > > > + goto err_media_unregister;
> > > > > > + }
> > > > > > +
> > > > > > + /* Enable the block power domain */
> > > > > > + pm_runtime_enable(&pdev->dev);
> > > > > > +
> > > > > > + return 0;
> > > > > > +
> > > > > > +err_media_unregister:
> > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > +err_v4l2_unregister:
> > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > +err_unicam_put:
> > > > > > + unicam_put(unicam);
> > > > > > +
> > > > > > + return ret;
> > > > > > +}
> > > > > > +
> > > > > > +static int unicam_remove(struct platform_device *pdev)
> > > > > > +{
> > > > > > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > > > > > +
> > > > > > + unicam_dbg(2, unicam, "%s\n", __func__);
> > > > > > +
> > > > > > + v4l2_async_notifier_unregister(&unicam->notifier);
> > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > + unregister_nodes(unicam);
> > > > > > +
> > > > > > + pm_runtime_disable(&pdev->dev);
> > > > > > +
> > > > > > + return 0;
> > > > > > +}
> > > > > > +
> > > > > > +static const struct of_device_id unicam_of_match[] = {
> > > > > > + { .compatible = "brcm,bcm2835-unicam", },
> > > > > > + { /* sentinel */ },
> > > > > > +};
> > > > > > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > > > > > +
> > > > > > +static struct platform_driver unicam_driver = {
> > > > > > + .probe = unicam_probe,
> > > > > > + .remove = unicam_remove,
> > > > > > + .driver = {
> > > > > > + .name = UNICAM_MODULE_NAME,
> > > > > > + .of_match_table = of_match_ptr(unicam_of_match),
> > > > > > + },
> > > > > > +};
> > > > > > +
> > > > > > +module_platform_driver(unicam_driver);
> > > > > > +
> > > > > > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > > > > > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > > > > > +MODULE_LICENSE("GPL");
> > > > > > +MODULE_VERSION(UNICAM_VERSION);
> > > > > > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > new file mode 100644
> > > > > > index 000000000000..ae059a171d0f
> > > > > > --- /dev/null
> > > > > > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > @@ -0,0 +1,253 @@
> > > > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > > > +
> > > > > > +/*
> > > > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > + */
> > > > > > +
> > > > > > +#ifndef VC4_REGS_UNICAM_H
> > > > > > +#define VC4_REGS_UNICAM_H
> > > > > > +
> > > > > > +/*
> > > > > > + * The following values are taken from files found within the code drop
> > > > > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > > > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > > > > + * They have been modified to be only the register offset.
> > > > > > + */
> > > > > > +#define UNICAM_CTRL 0x000
> > > > > > +#define UNICAM_STA 0x004
> > > > > > +#define UNICAM_ANA 0x008
> > > > > > +#define UNICAM_PRI 0x00c
> > > > > > +#define UNICAM_CLK 0x010
> > > > > > +#define UNICAM_CLT 0x014
> > > > > > +#define UNICAM_DAT0 0x018
> > > > > > +#define UNICAM_DAT1 0x01c
> > > > > > +#define UNICAM_DAT2 0x020
> > > > > > +#define UNICAM_DAT3 0x024
> > > > > > +#define UNICAM_DLT 0x028
> > > > > > +#define UNICAM_CMP0 0x02c
> > > > > > +#define UNICAM_CMP1 0x030
> > > > > > +#define UNICAM_CAP0 0x034
> > > > > > +#define UNICAM_CAP1 0x038
> > > > > > +#define UNICAM_ICTL 0x100
> > > > > > +#define UNICAM_ISTA 0x104
> > > > > > +#define UNICAM_IDI0 0x108
> > > > > > +#define UNICAM_IPIPE 0x10c
> > > > > > +#define UNICAM_IBSA0 0x110
> > > > > > +#define UNICAM_IBEA0 0x114
> > > > > > +#define UNICAM_IBLS 0x118
> > > > > > +#define UNICAM_IBWP 0x11c
> > > > > > +#define UNICAM_IHWIN 0x120
> > > > > > +#define UNICAM_IHSTA 0x124
> > > > > > +#define UNICAM_IVWIN 0x128
> > > > > > +#define UNICAM_IVSTA 0x12c
> > > > > > +#define UNICAM_ICC 0x130
> > > > > > +#define UNICAM_ICS 0x134
> > > > > > +#define UNICAM_IDC 0x138
> > > > > > +#define UNICAM_IDPO 0x13c
> > > > > > +#define UNICAM_IDCA 0x140
> > > > > > +#define UNICAM_IDCD 0x144
> > > > > > +#define UNICAM_IDS 0x148
> > > > > > +#define UNICAM_DCS 0x200
> > > > > > +#define UNICAM_DBSA0 0x204
> > > > > > +#define UNICAM_DBEA0 0x208
> > > > > > +#define UNICAM_DBWP 0x20c
> > > > > > +#define UNICAM_DBCTL 0x300
> > > > > > +#define UNICAM_IBSA1 0x304
> > > > > > +#define UNICAM_IBEA1 0x308
> > > > > > +#define UNICAM_IDI1 0x30c
> > > > > > +#define UNICAM_DBSA1 0x310
> > > > > > +#define UNICAM_DBEA1 0x314
> > > > > > +#define UNICAM_MISC 0x400
> > > > > > +
> > > > > > +/*
> > > > > > + * The following bitmasks are from the kernel released by Broadcom
> > > > > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > > > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > > > > + * Unicam block as BCM2835, as defined in eg
> > > > > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > > > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > > > > + *
> > > > > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > > > > + */
> > > > > > +/* UNICAM_CTRL Register */
> > > > > > +#define UNICAM_CPE BIT(0)
> > > > > > +#define UNICAM_MEM BIT(1)
> > > > > > +#define UNICAM_CPR BIT(2)
> > > > > > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > > > > > +#define UNICAM_CPM_CSI2 0
> > > > > > +#define UNICAM_CPM_CCP2 1
> > > > > > +#define UNICAM_SOE BIT(4)
> > > > > > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > > > > > +#define UNICAM_DCM_STROBE 0
> > > > > > +#define UNICAM_DCM_DATA 1
> > > > > > +#define UNICAM_SLS BIT(6)
> > > > > > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > > > > > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > > > > > +
> > > > > > +/* UNICAM_STA Register */
> > > > > > +#define UNICAM_SYN BIT(0)
> > > > > > +#define UNICAM_CS BIT(1)
> > > > > > +#define UNICAM_SBE BIT(2)
> > > > > > +#define UNICAM_PBE BIT(3)
> > > > > > +#define UNICAM_HOE BIT(4)
> > > > > > +#define UNICAM_PLE BIT(5)
> > > > > > +#define UNICAM_SSC BIT(6)
> > > > > > +#define UNICAM_CRCE BIT(7)
> > > > > > +#define UNICAM_OES BIT(8)
> > > > > > +#define UNICAM_IFO BIT(9)
> > > > > > +#define UNICAM_OFO BIT(10)
> > > > > > +#define UNICAM_BFO BIT(11)
> > > > > > +#define UNICAM_DL BIT(12)
> > > > > > +#define UNICAM_PS BIT(13)
> > > > > > +#define UNICAM_IS BIT(14)
> > > > > > +#define UNICAM_PI0 BIT(15)
> > > > > > +#define UNICAM_PI1 BIT(16)
> > > > > > +#define UNICAM_FSI_S BIT(17)
> > > > > > +#define UNICAM_FEI_S BIT(18)
> > > > > > +#define UNICAM_LCI_S BIT(19)
> > > > > > +#define UNICAM_BUF0_RDY BIT(20)
> > > > > > +#define UNICAM_BUF0_NO BIT(21)
> > > > > > +#define UNICAM_BUF1_RDY BIT(22)
> > > > > > +#define UNICAM_BUF1_NO BIT(23)
> > > > > > +#define UNICAM_DI BIT(24)
> > > > > > +
> > > > > > +#define UNICAM_STA_MASK_ALL \
> > > > > > + (UNICAM_DL + \
> > > > > > + UNICAM_SBE + \
> > > > > > + UNICAM_PBE + \
> > > > > > + UNICAM_HOE + \
> > > > > > + UNICAM_PLE + \
> > > > > > + UNICAM_SSC + \
> > > > > > + UNICAM_CRCE + \
> > > > > > + UNICAM_IFO + \
> > > > > > + UNICAM_OFO + \
> > > > > > + UNICAM_PS + \
> > > > > > + UNICAM_PI0 + \
> > > > > > + UNICAM_PI1)
> > > > > > +
> > > > > > +/* UNICAM_ANA Register */
> > > > > > +#define UNICAM_APD BIT(0)
> > > > > > +#define UNICAM_BPD BIT(1)
> > > > > > +#define UNICAM_AR BIT(2)
> > > > > > +#define UNICAM_DDL BIT(3)
> > > > > > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > > > > > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > > > > > +
> > > > > > +/* UNICAM_PRI Register */
> > > > > > +#define UNICAM_PE BIT(0)
> > > > > > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > > > > > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > > > > > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > > > > > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > > > > > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > > > > > +
> > > > > > +/* UNICAM_CLK Register */
> > > > > > +#define UNICAM_CLE BIT(0)
> > > > > > +#define UNICAM_CLPD BIT(1)
> > > > > > +#define UNICAM_CLLPE BIT(2)
> > > > > > +#define UNICAM_CLHSE BIT(3)
> > > > > > +#define UNICAM_CLTRE BIT(4)
> > > > > > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > > > > > +#define UNICAM_CLSTE BIT(29)
> > > > > > +
> > > > > > +/* UNICAM_CLT Register */
> > > > > > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > > > > > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > > > > > +
> > > > > > +/* UNICAM_DATn Registers */
> > > > > > +#define UNICAM_DLE BIT(0)
> > > > > > +#define UNICAM_DLPD BIT(1)
> > > > > > +#define UNICAM_DLLPE BIT(2)
> > > > > > +#define UNICAM_DLHSE BIT(3)
> > > > > > +#define UNICAM_DLTRE BIT(4)
> > > > > > +#define UNICAM_DLSM BIT(5)
> > > > > > +#define UNICAM_DLFO BIT(28)
> > > > > > +#define UNICAM_DLSTE BIT(29)
> > > > > > +
> > > > > > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > > > > > +
> > > > > > +/* UNICAM_DLT Register */
> > > > > > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > > > > > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > > > > > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > > > > > +
> > > > > > +/* UNICAM_ICTL Register */
> > > > > > +#define UNICAM_FSIE BIT(0)
> > > > > > +#define UNICAM_FEIE BIT(1)
> > > > > > +#define UNICAM_IBOB BIT(2)
> > > > > > +#define UNICAM_FCM BIT(3)
> > > > > > +#define UNICAM_TFC BIT(4)
> > > > > > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > > > > > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > > > > > +
> > > > > > +/* UNICAM_IDI0/1 Register */
> > > > > > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > > > > > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > > > > > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > > > > > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > > > > > +
> > > > > > +/* UNICAM_ISTA Register */
> > > > > > +#define UNICAM_FSI BIT(0)
> > > > > > +#define UNICAM_FEI BIT(1)
> > > > > > +#define UNICAM_LCI BIT(2)
> > > > > > +
> > > > > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > > > > > +
> > > > > > +/* UNICAM_IPIPE Register */
> > > > > > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > > > > > + /* Unpacking modes */
> > > > > > + #define UNICAM_PUM_NONE 0
> > > > > > + #define UNICAM_PUM_UNPACK6 1
> > > > > > + #define UNICAM_PUM_UNPACK7 2
> > > > > > + #define UNICAM_PUM_UNPACK8 3
> > > > > > + #define UNICAM_PUM_UNPACK10 4
> > > > > > + #define UNICAM_PUM_UNPACK12 5
> > > > > > + #define UNICAM_PUM_UNPACK14 6
> > > > > > + #define UNICAM_PUM_UNPACK16 7
> > > > > > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > > > > > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > > > > > + /* Packing modes */
> > > > > > + #define UNICAM_PPM_NONE 0
> > > > > > + #define UNICAM_PPM_PACK8 1
> > > > > > + #define UNICAM_PPM_PACK10 2
> > > > > > + #define UNICAM_PPM_PACK12 3
> > > > > > + #define UNICAM_PPM_PACK14 4
> > > > > > + #define UNICAM_PPM_PACK16 5
> > > > > > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > > > > > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > > > > > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > > > > > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > > > > > +
> > > > > > +/* UNICAM_ICC Register */
> > > > > > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > > > > > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > > > > > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > > > > > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > > > > > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > > > > > +
> > > > > > +/* UNICAM_DCS Register */
> > > > > > +#define UNICAM_DIE BIT(0)
> > > > > > +#define UNICAM_DIM BIT(1)
> > > > > > +#define UNICAM_DBOB BIT(3)
> > > > > > +#define UNICAM_FDE BIT(4)
> > > > > > +#define UNICAM_LDP BIT(5)
> > > > > > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > > > > > +
> > > > > > +/* UNICAM_DBCTL Register */
> > > > > > +#define UNICAM_DBEN BIT(0)
> > > > > > +#define UNICAM_BUF0_IE BIT(1)
> > > > > > +#define UNICAM_BUF1_IE BIT(2)
> > > > > > +
> > > > > > +/* UNICAM_CMP[0,1] register */
> > > > > > +#define UNICAM_PCE BIT(31)
> > > > > > +#define UNICAM_GI BIT(9)
> > > > > > +#define UNICAM_CPH BIT(8)
> > > > > > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > > > > > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > > > > > +
> > > > > > +/* UNICAM_MISC register */
> > > > > > +#define UNICAM_FL0 BIT(6)
> > > > > > +#define UNICAM_FL1 BIT(9)
> > > > > > +
> > > > > > +#endif
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-08-31 14:56 ` Laurent Pinchart
@ 2020-09-01 8:41 ` Dave Stevenson
2020-09-01 10:22 ` Jacopo Mondi
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-09-01 8:41 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Jacopo Mondi, Linux Media Mailing List, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
Morning Laurent and Jacopo.
On Mon, 31 Aug 2020 at 15:57, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Jacopo,
>
> On Mon, Aug 31, 2020 at 04:46:37PM +0200, Jacopo Mondi wrote:
> > On Mon, Aug 31, 2020 at 05:17:04PM +0300, Laurent Pinchart wrote:
> > > On Mon, Aug 31, 2020 at 09:38:58AM +0200, Jacopo Mondi wrote:
> > > > On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> > > > > On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> > > > > > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > > > > > From: Naushir Patuck <naush@raspberrypi.com>
> > > > > > >
> > > > > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > > > > Compared to the bcm2835-camera driver present in staging, this driver
> > > > > > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > > > > > the VC4 firmware running on the VPU.
> > > > > > >
> > > > > > > The commit is made up of a series of changes cherry-picked from the
> > > > > > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > > > > > additional enhancements, forward-ported to the mainline kernel.
> > > > > > >
> > > > > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > > > > ---
> > > > > > > Changes since v1:
> > > > > > >
> > > > > > > - Re-fetch mbus code from subdev on a g_fmt call
> > > > > > > - Group all ioctl disabling together
> > > > > > > - Fix reference counting in unicam_open
> > > > > > > - Add support for VIDIOC_[S|G]_SELECTION
> > > > > > > ---
> > > > > >
> > > > > > [snip]
> > > > > >
> > > > > > > +
> > > > > > > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > > > > > > +{
> > > > > > > + struct platform_device *pdev = dev->pdev;
> > > > > > > + struct v4l2_fwnode_endpoint ep = { 0 };
> > > > > > > + struct device_node *ep_node;
> > > > > > > + struct device_node *sensor_node;
> > > > > > > + unsigned int lane;
> > > > > > > + int ret = -EINVAL;
> > > > > > > +
> > > > > > > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > > > > > > + &dev->max_data_lanes) < 0) {
> > > > > > > + unicam_err(dev, "number of data lanes not set\n");
> > > > > > > + return -EINVAL;
> > > > > > > + }
> > > > > >
> > > > > > mmm, this seems like a good use case for the newly introduced
> > > > > > get_mbus_config pad operation, isn't it ?
> > > > > >
> > > > > > We could drop the custom "brcm,num-data-lanes" property, use the
> > > > > > standard "num-lanes" to describe the number of wired data lanes, and
> > > > > > get the number of actually used ones at s_stream(1) time with the new pad
> > > > > > operation
> > > > > >
> > > > > > What do you think ? This would require implemeting the operation in
> > > > > > each subdev the UNICAM interface has to work with, but this doesn't
> > > > > > sound too much of a work.
> > > > > >
> > > > > > For reference this is how it happens on rcar+adv748x:
> > > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> > > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
> > > > >
> > > > > Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> > > > > The former tells how many data lanes the unicam IP has (the same IP,
> > > > > with the same compatible string, is instantiated in different SoCs with
> > > > > different number of lanes), while the latter queries at runtime how many
> > > > > data lanes the remote subdev will use for its current configuration. The
> > > >
> > > > Can't we get the number of available data lanes just by parsing the
> > > > endpoint ?
> > >
> > > We can get the number of data lanes routed on the board, which may not
> > > match the number of data lanes available in the IP. I'll let Dave
> > > comment on this, I recall we had a lengthy discussion on this topic in
> > > the past, but I don't remember the details :-S
Thread "[PATCH 10/10] media: bcm2835-unicam: Simplify OF parsing"
posted to the private libcamera list Sun, 19 Apr. I can forward on if
others have deleted it.
> > I think the max number of number of lanes in the IP should come in DT
> > by restricting the number of accepted data-lanes value depending on
> > the compatible string.
>
> Unless I'm mistaken we have a single compatibility string.
Currently we have a single compatible string.
So you want a "brcm,bcm2835-unicam-2-lane" and "brcm,bcm2835-unicam-4-lane"?
You can't guarantee that unicam0 is always 2 lane and unicam1 is
always 4 lane forever more - I know there are Broadcom chips with a
unicam2 for a start.
I defer to the DT gatekeepers, but that feels ugly.
> > > > I'm aware of the implication that the 'data-lanes' property
> > > > would differ in the bridge and sensor endpoints, but now that we have
> > > > a mechanism to get that information wihtout parsing the remote
> > > > endpoint it doesn't seem wrong to me.
> > >
> > > That's not the issue at hand. The data-lanes property need to match on
> > > both sides of a link, as it tells how many data lanes are routed on the
> > > board. Of course that becomes problematic when we start using overlays
> > > for pluggable boards, as the two sides can be designed with a different
> > > number of routed lanes.
> >
> > It actually is. As I read the current implementation, UNICAM has 4
> > lanes available, 2 are enabled in the endpoint, and the remote uses 2.
>
> That's because the board has 2 data lanes routed.
>
> > If we set the number of lanes in the endpoint to 4 (the actually
> > available ones) and use only the two required by querying the remote
> > end at s_stream time, we don't need any custom sourcery
>
> That would be incorrect, as there are two data lanes routed, so you
> shouldn't set the data-lanes property to 4 lanes in the endpoint.
>
> > > > And that's not different from what it's done in R-Car (half-true: we
> > > > have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
> > > > as you said the number of active data lanes can be downscaled to
> > > > transmit low resolution streams).
> > > >
> > > > I think Dave already had this conversation in the past, I might dig
> > > > the archive and search for a reference.
Referenced above.
> > > > > DT property serves as a sanity check, to ensure to no more than the
> > > > > number of supported data lanes is used, while the subdev operation
> > > > > should be used in the unicam driver to support sources that will
> > > > > transmit over a different number of data lanes depending on the
> > > > > configuration (mostly depending on the resolution). We don't have to
> > > >
> > > > I think reading the remote's mbus configuration might also help in
> > > > support connecting pluggable devices with a different number of wired
> > > > data lanes than the bridge's available ones.
> > >
> > > No, that we shouldn't do. DT properties need to be interpreted within
> > > the context of a compatible string, reading properties of a remote node
> > > is a no-go in most cases. It's really really something that must not be
> > > done without careful consideration, and is never a simple way to work
> > > around a problem.
> >
> > What I mean is that get_mbus_config() allows you to retrieve those
> > information -without- having to read the remote endpoint, which has
> > always been a blocker for this kind of operations (and I agree it
> > shouldn't be done).
I like get_mbus_config as far as it goes.
When it last came up I mentioned that it doesn't convey things like
the continuous clock setting, so you have to have the DT endpoints
configured identically for that to avoid issues, and if you start to
involve CSI2 mux chips then both sources have to have the same
requirements.
Personally I'd love to see get_mbus_config being the correct way of
configuring all receiver parameters as you'd then be guaranteed to
have source and receiver matching. Drop back to the receiver's DT
endpoint when get_mbus_config isn't supported.
> > > > Am I missing something maybe? The non-connected lanes on the bride
> > > > will simply stay unused, can't they ?
> > > >
> > > > > implement support for get_mbus_config just yet, it can be delayed until
> > > > > we have a source that requires it.
> > > >
> > > > I sketched a few patches yesterday basically appling what's done in
> > > > 5cef0785362ed and a61b1b5d41735. Is a custom property better in your
> > > > opinion ?
> > >
> > > Again, the custom property only specifies the number of data lanes
> > > implemented in the unicam IP core. It's nothing else than that, and does
> > > *not* convey any information about how many data lanes the sensor will
> > > use.
> >
> > It shouldn't, and to me that information should come from DT as we do
> > for all other platforms.
> >
> > Let's see, if feels a bit a waste to use a custom property for
> > something we can do without, and usage of get_mbus_config() would
> > allow to support switching devices at run-time with a different number
> > of data-lanes, as use case I know Dave has downstream.
>
> This is *still* not what brcm,num-lanes is about... get_mbus_config()
> will not tell you how many data lanes the unicam IP supports.
Use case of a TC358743 HDMI to CSI2 bridge connected to Unicam0 (2 lane).
TC358743 supports get_mbus_config and says it wants 4 lanes for 1080p60 RGB888.
Unicam calls get_mbus_config and gets told 4.
So how do we detect and reject that invalid configuration? It can't
rely on the Unicam data-lanes as that is supposed to reflect the
number of active lanes, but it's not something that can be otherwise
detected and blocked. Start streaming and it'll just sit there dumbly
doing nothing.
Yes we did discuss that most of the Pi boards have only 2 of the 4
data lanes from Unicam1 wired out to the camera connector, so whilst
technically not what the binding says, it is possible to then drop
num-data-lanes down to 2 (though the IP has 4 lanes) and get
validation of the config in that case as well.
Dave
> > > > > > +
> > > > > > > + /* Get the local endpoint and remote device. */
> > > > > > > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > > > > > > + if (!ep_node) {
> > > > > > > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > > > > > > + return -EINVAL;
> > > > > > > + }
> > > > > > > +
> > > > > > > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > > > > > > +
> > > > > > > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > > > > > > + if (!sensor_node) {
> > > > > > > + unicam_dbg(3, dev, "can't get remote parent\n");
> > > > > > > + goto cleanup_exit;
> > > > > > > + }
> > > > > > > +
> > > > > > > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > > > > > > +
> > > > > > > + /* Parse the local endpoint and validate its configuration. */
> > > > > > > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > > > > > > +
> > > > > > > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > > > > > > + ep.bus_type);
> > > > > > > +
> > > > > > > + dev->bus_type = ep.bus_type;
> > > > > > > +
> > > > > > > + switch (ep.bus_type) {
> > > > > > > + case V4L2_MBUS_CSI2_DPHY:
> > > > > > > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > > > > > > + case 1:
> > > > > > > + case 2:
> > > > > > > + case 4:
> > > > > > > + break;
> > > > > > > +
> > > > > > > + default:
> > > > > > > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > > > > > > + sensor_node,
> > > > > > > + ep.bus.mipi_csi2.num_data_lanes);
> > > > > > > + goto cleanup_exit;
> > > > > > > + }
> > > > > > > +
> > > > > > > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > > > > > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > > > > > > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > > > > > > + sensor_node);
> > > > > > > + goto cleanup_exit;
> > > > > > > + }
> > > > > > > + }
> > > > > > > +
> > > > > > > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > > > > > > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > > > > > > + ep.bus.mipi_csi2.num_data_lanes,
> > > > > > > + dev->max_data_lanes);
> > > > > > > + }
> > > > > > > +
> > > > > > > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > > > > > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > > > > > > +
> > > > > > > + break;
> > > > > > > +
> > > > > > > + case V4L2_MBUS_CCP2:
> > > > > > > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > > > > > > + ep.bus.mipi_csi1.data_lane != 1) {
> > > > > > > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > > > > > > + sensor_node);
> > > > > > > + goto cleanup_exit;
> > > > > > > + }
> > > > > > > +
> > > > > > > + dev->max_data_lanes = 1;
> > > > > > > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > > > > > > + break;
> > > > > > > +
> > > > > > > + default:
> > > > > > > + /* Unsupported bus type */
> > > > > > > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > > > > > > + sensor_node, ep.bus_type);
> > > > > > > + goto cleanup_exit;
> > > > > > > + }
> > > > > > > +
> > > > > > > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > > > > > > + sensor_node,
> > > > > > > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > > > > > > + dev->max_data_lanes, dev->bus_flags);
> > > > > > > +
> > > > > > > + /* Initialize and register the async notifier. */
> > > > > > > + v4l2_async_notifier_init(&dev->notifier);
> > > > > > > + dev->notifier.ops = &unicam_async_ops;
> > > > > > > +
> > > > > > > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > > > > > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > > > > > > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > > > > > > + if (ret) {
> > > > > > > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > > > > > > + goto cleanup_exit;
> > > > > > > + }
> > > > > > > +
> > > > > > > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > > > > > > + if (ret) {
> > > > > > > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > > > > > > + ret = -EINVAL;
> > > > > > > + }
> > > > > > > +
> > > > > > > +cleanup_exit:
> > > > > > > + of_node_put(sensor_node);
> > > > > > > + of_node_put(ep_node);
> > > > > > > +
> > > > > > > + return ret;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static int unicam_probe(struct platform_device *pdev)
> > > > > > > +{
> > > > > > > + struct unicam_device *unicam;
> > > > > > > + int ret;
> > > > > > > +
> > > > > > > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > > > > > > + if (!unicam)
> > > > > > > + return -ENOMEM;
> > > > > > > +
> > > > > > > + kref_init(&unicam->kref);
> > > > > > > + unicam->pdev = pdev;
> > > > > > > +
> > > > > > > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > > > > > > + if (IS_ERR(unicam->base)) {
> > > > > > > + unicam_err(unicam, "Failed to get main io block\n");
> > > > > > > + ret = PTR_ERR(unicam->base);
> > > > > > > + goto err_unicam_put;
> > > > > > > + }
> > > > > > > +
> > > > > > > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > > > > > > + if (IS_ERR(unicam->clk_gate_base)) {
> > > > > > > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > > > > > > + ret = PTR_ERR(unicam->clk_gate_base);
> > > > > > > + goto err_unicam_put;
> > > > > > > + }
> > > > > > > +
> > > > > > > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > > > > > > + if (IS_ERR(unicam->clock)) {
> > > > > > > + unicam_err(unicam, "Failed to get clock\n");
> > > > > > > + ret = PTR_ERR(unicam->clock);
> > > > > > > + goto err_unicam_put;
> > > > > > > + }
> > > > > > > +
> > > > > > > + ret = platform_get_irq(pdev, 0);
> > > > > > > + if (ret <= 0) {
> > > > > > > + dev_err(&pdev->dev, "No IRQ resource\n");
> > > > > > > + ret = -EINVAL;
> > > > > > > + goto err_unicam_put;
> > > > > > > + }
> > > > > > > +
> > > > > > > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > > > > > > + "unicam_capture0", unicam);
> > > > > > > + if (ret) {
> > > > > > > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > > > > > > + ret = -EINVAL;
> > > > > > > + goto err_unicam_put;
> > > > > > > + }
> > > > > > > +
> > > > > > > + unicam->mdev.dev = &pdev->dev;
> > > > > > > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > > > > > + sizeof(unicam->mdev.model));
> > > > > > > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > > > > > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > > > > > > + "platform:%s", dev_name(&pdev->dev));
> > > > > > > + unicam->mdev.hw_revision = 0;
> > > > > > > +
> > > > > > > + media_device_init(&unicam->mdev);
> > > > > > > +
> > > > > > > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > > > > > > +
> > > > > > > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > > > > > > + if (ret) {
> > > > > > > + unicam_err(unicam,
> > > > > > > + "Unable to register v4l2 device.\n");
> > > > > > > + goto err_unicam_put;
> > > > > > > + }
> > > > > > > +
> > > > > > > + ret = media_device_register(&unicam->mdev);
> > > > > > > + if (ret < 0) {
> > > > > > > + unicam_err(unicam,
> > > > > > > + "Unable to register media-controller device.\n");
> > > > > > > + goto err_v4l2_unregister;
> > > > > > > + }
> > > > > > > +
> > > > > > > + /* Reserve space for the controls */
> > > > > > > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > > > > > > + if (ret < 0)
> > > > > > > + goto err_media_unregister;
> > > > > > > +
> > > > > > > + /* set the driver data in platform device */
> > > > > > > + platform_set_drvdata(pdev, unicam);
> > > > > > > +
> > > > > > > + ret = of_unicam_connect_subdevs(unicam);
> > > > > > > + if (ret) {
> > > > > > > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > > > > > > + goto err_media_unregister;
> > > > > > > + }
> > > > > > > +
> > > > > > > + /* Enable the block power domain */
> > > > > > > + pm_runtime_enable(&pdev->dev);
> > > > > > > +
> > > > > > > + return 0;
> > > > > > > +
> > > > > > > +err_media_unregister:
> > > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > > +err_v4l2_unregister:
> > > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > > +err_unicam_put:
> > > > > > > + unicam_put(unicam);
> > > > > > > +
> > > > > > > + return ret;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static int unicam_remove(struct platform_device *pdev)
> > > > > > > +{
> > > > > > > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > > > > > > +
> > > > > > > + unicam_dbg(2, unicam, "%s\n", __func__);
> > > > > > > +
> > > > > > > + v4l2_async_notifier_unregister(&unicam->notifier);
> > > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > > + unregister_nodes(unicam);
> > > > > > > +
> > > > > > > + pm_runtime_disable(&pdev->dev);
> > > > > > > +
> > > > > > > + return 0;
> > > > > > > +}
> > > > > > > +
> > > > > > > +static const struct of_device_id unicam_of_match[] = {
> > > > > > > + { .compatible = "brcm,bcm2835-unicam", },
> > > > > > > + { /* sentinel */ },
> > > > > > > +};
> > > > > > > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > > > > > > +
> > > > > > > +static struct platform_driver unicam_driver = {
> > > > > > > + .probe = unicam_probe,
> > > > > > > + .remove = unicam_remove,
> > > > > > > + .driver = {
> > > > > > > + .name = UNICAM_MODULE_NAME,
> > > > > > > + .of_match_table = of_match_ptr(unicam_of_match),
> > > > > > > + },
> > > > > > > +};
> > > > > > > +
> > > > > > > +module_platform_driver(unicam_driver);
> > > > > > > +
> > > > > > > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > > > > > > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > > > > > > +MODULE_LICENSE("GPL");
> > > > > > > +MODULE_VERSION(UNICAM_VERSION);
> > > > > > > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > > new file mode 100644
> > > > > > > index 000000000000..ae059a171d0f
> > > > > > > --- /dev/null
> > > > > > > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > > @@ -0,0 +1,253 @@
> > > > > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > > > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > > + */
> > > > > > > +
> > > > > > > +#ifndef VC4_REGS_UNICAM_H
> > > > > > > +#define VC4_REGS_UNICAM_H
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * The following values are taken from files found within the code drop
> > > > > > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > > > > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > > > > > + * They have been modified to be only the register offset.
> > > > > > > + */
> > > > > > > +#define UNICAM_CTRL 0x000
> > > > > > > +#define UNICAM_STA 0x004
> > > > > > > +#define UNICAM_ANA 0x008
> > > > > > > +#define UNICAM_PRI 0x00c
> > > > > > > +#define UNICAM_CLK 0x010
> > > > > > > +#define UNICAM_CLT 0x014
> > > > > > > +#define UNICAM_DAT0 0x018
> > > > > > > +#define UNICAM_DAT1 0x01c
> > > > > > > +#define UNICAM_DAT2 0x020
> > > > > > > +#define UNICAM_DAT3 0x024
> > > > > > > +#define UNICAM_DLT 0x028
> > > > > > > +#define UNICAM_CMP0 0x02c
> > > > > > > +#define UNICAM_CMP1 0x030
> > > > > > > +#define UNICAM_CAP0 0x034
> > > > > > > +#define UNICAM_CAP1 0x038
> > > > > > > +#define UNICAM_ICTL 0x100
> > > > > > > +#define UNICAM_ISTA 0x104
> > > > > > > +#define UNICAM_IDI0 0x108
> > > > > > > +#define UNICAM_IPIPE 0x10c
> > > > > > > +#define UNICAM_IBSA0 0x110
> > > > > > > +#define UNICAM_IBEA0 0x114
> > > > > > > +#define UNICAM_IBLS 0x118
> > > > > > > +#define UNICAM_IBWP 0x11c
> > > > > > > +#define UNICAM_IHWIN 0x120
> > > > > > > +#define UNICAM_IHSTA 0x124
> > > > > > > +#define UNICAM_IVWIN 0x128
> > > > > > > +#define UNICAM_IVSTA 0x12c
> > > > > > > +#define UNICAM_ICC 0x130
> > > > > > > +#define UNICAM_ICS 0x134
> > > > > > > +#define UNICAM_IDC 0x138
> > > > > > > +#define UNICAM_IDPO 0x13c
> > > > > > > +#define UNICAM_IDCA 0x140
> > > > > > > +#define UNICAM_IDCD 0x144
> > > > > > > +#define UNICAM_IDS 0x148
> > > > > > > +#define UNICAM_DCS 0x200
> > > > > > > +#define UNICAM_DBSA0 0x204
> > > > > > > +#define UNICAM_DBEA0 0x208
> > > > > > > +#define UNICAM_DBWP 0x20c
> > > > > > > +#define UNICAM_DBCTL 0x300
> > > > > > > +#define UNICAM_IBSA1 0x304
> > > > > > > +#define UNICAM_IBEA1 0x308
> > > > > > > +#define UNICAM_IDI1 0x30c
> > > > > > > +#define UNICAM_DBSA1 0x310
> > > > > > > +#define UNICAM_DBEA1 0x314
> > > > > > > +#define UNICAM_MISC 0x400
> > > > > > > +
> > > > > > > +/*
> > > > > > > + * The following bitmasks are from the kernel released by Broadcom
> > > > > > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > > > > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > > > > > + * Unicam block as BCM2835, as defined in eg
> > > > > > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > > > > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > > > > > + *
> > > > > > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > > > > > + */
> > > > > > > +/* UNICAM_CTRL Register */
> > > > > > > +#define UNICAM_CPE BIT(0)
> > > > > > > +#define UNICAM_MEM BIT(1)
> > > > > > > +#define UNICAM_CPR BIT(2)
> > > > > > > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > > > > > > +#define UNICAM_CPM_CSI2 0
> > > > > > > +#define UNICAM_CPM_CCP2 1
> > > > > > > +#define UNICAM_SOE BIT(4)
> > > > > > > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > > > > > > +#define UNICAM_DCM_STROBE 0
> > > > > > > +#define UNICAM_DCM_DATA 1
> > > > > > > +#define UNICAM_SLS BIT(6)
> > > > > > > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > > > > > > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > > > > > > +
> > > > > > > +/* UNICAM_STA Register */
> > > > > > > +#define UNICAM_SYN BIT(0)
> > > > > > > +#define UNICAM_CS BIT(1)
> > > > > > > +#define UNICAM_SBE BIT(2)
> > > > > > > +#define UNICAM_PBE BIT(3)
> > > > > > > +#define UNICAM_HOE BIT(4)
> > > > > > > +#define UNICAM_PLE BIT(5)
> > > > > > > +#define UNICAM_SSC BIT(6)
> > > > > > > +#define UNICAM_CRCE BIT(7)
> > > > > > > +#define UNICAM_OES BIT(8)
> > > > > > > +#define UNICAM_IFO BIT(9)
> > > > > > > +#define UNICAM_OFO BIT(10)
> > > > > > > +#define UNICAM_BFO BIT(11)
> > > > > > > +#define UNICAM_DL BIT(12)
> > > > > > > +#define UNICAM_PS BIT(13)
> > > > > > > +#define UNICAM_IS BIT(14)
> > > > > > > +#define UNICAM_PI0 BIT(15)
> > > > > > > +#define UNICAM_PI1 BIT(16)
> > > > > > > +#define UNICAM_FSI_S BIT(17)
> > > > > > > +#define UNICAM_FEI_S BIT(18)
> > > > > > > +#define UNICAM_LCI_S BIT(19)
> > > > > > > +#define UNICAM_BUF0_RDY BIT(20)
> > > > > > > +#define UNICAM_BUF0_NO BIT(21)
> > > > > > > +#define UNICAM_BUF1_RDY BIT(22)
> > > > > > > +#define UNICAM_BUF1_NO BIT(23)
> > > > > > > +#define UNICAM_DI BIT(24)
> > > > > > > +
> > > > > > > +#define UNICAM_STA_MASK_ALL \
> > > > > > > + (UNICAM_DL + \
> > > > > > > + UNICAM_SBE + \
> > > > > > > + UNICAM_PBE + \
> > > > > > > + UNICAM_HOE + \
> > > > > > > + UNICAM_PLE + \
> > > > > > > + UNICAM_SSC + \
> > > > > > > + UNICAM_CRCE + \
> > > > > > > + UNICAM_IFO + \
> > > > > > > + UNICAM_OFO + \
> > > > > > > + UNICAM_PS + \
> > > > > > > + UNICAM_PI0 + \
> > > > > > > + UNICAM_PI1)
> > > > > > > +
> > > > > > > +/* UNICAM_ANA Register */
> > > > > > > +#define UNICAM_APD BIT(0)
> > > > > > > +#define UNICAM_BPD BIT(1)
> > > > > > > +#define UNICAM_AR BIT(2)
> > > > > > > +#define UNICAM_DDL BIT(3)
> > > > > > > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > > > > > > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > > > > > > +
> > > > > > > +/* UNICAM_PRI Register */
> > > > > > > +#define UNICAM_PE BIT(0)
> > > > > > > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > > > > > > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > > > > > > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > > > > > > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > > > > > > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > > > > > > +
> > > > > > > +/* UNICAM_CLK Register */
> > > > > > > +#define UNICAM_CLE BIT(0)
> > > > > > > +#define UNICAM_CLPD BIT(1)
> > > > > > > +#define UNICAM_CLLPE BIT(2)
> > > > > > > +#define UNICAM_CLHSE BIT(3)
> > > > > > > +#define UNICAM_CLTRE BIT(4)
> > > > > > > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > > > > > > +#define UNICAM_CLSTE BIT(29)
> > > > > > > +
> > > > > > > +/* UNICAM_CLT Register */
> > > > > > > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > > > > > > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > > > > > > +
> > > > > > > +/* UNICAM_DATn Registers */
> > > > > > > +#define UNICAM_DLE BIT(0)
> > > > > > > +#define UNICAM_DLPD BIT(1)
> > > > > > > +#define UNICAM_DLLPE BIT(2)
> > > > > > > +#define UNICAM_DLHSE BIT(3)
> > > > > > > +#define UNICAM_DLTRE BIT(4)
> > > > > > > +#define UNICAM_DLSM BIT(5)
> > > > > > > +#define UNICAM_DLFO BIT(28)
> > > > > > > +#define UNICAM_DLSTE BIT(29)
> > > > > > > +
> > > > > > > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > > > > > > +
> > > > > > > +/* UNICAM_DLT Register */
> > > > > > > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > > > > > > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > > > > > > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > > > > > > +
> > > > > > > +/* UNICAM_ICTL Register */
> > > > > > > +#define UNICAM_FSIE BIT(0)
> > > > > > > +#define UNICAM_FEIE BIT(1)
> > > > > > > +#define UNICAM_IBOB BIT(2)
> > > > > > > +#define UNICAM_FCM BIT(3)
> > > > > > > +#define UNICAM_TFC BIT(4)
> > > > > > > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > > > > > > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > > > > > > +
> > > > > > > +/* UNICAM_IDI0/1 Register */
> > > > > > > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > > > > > > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > > > > > > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > > > > > > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > > > > > > +
> > > > > > > +/* UNICAM_ISTA Register */
> > > > > > > +#define UNICAM_FSI BIT(0)
> > > > > > > +#define UNICAM_FEI BIT(1)
> > > > > > > +#define UNICAM_LCI BIT(2)
> > > > > > > +
> > > > > > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > > > > > > +
> > > > > > > +/* UNICAM_IPIPE Register */
> > > > > > > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > > > > > > + /* Unpacking modes */
> > > > > > > + #define UNICAM_PUM_NONE 0
> > > > > > > + #define UNICAM_PUM_UNPACK6 1
> > > > > > > + #define UNICAM_PUM_UNPACK7 2
> > > > > > > + #define UNICAM_PUM_UNPACK8 3
> > > > > > > + #define UNICAM_PUM_UNPACK10 4
> > > > > > > + #define UNICAM_PUM_UNPACK12 5
> > > > > > > + #define UNICAM_PUM_UNPACK14 6
> > > > > > > + #define UNICAM_PUM_UNPACK16 7
> > > > > > > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > > > > > > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > > > > > > + /* Packing modes */
> > > > > > > + #define UNICAM_PPM_NONE 0
> > > > > > > + #define UNICAM_PPM_PACK8 1
> > > > > > > + #define UNICAM_PPM_PACK10 2
> > > > > > > + #define UNICAM_PPM_PACK12 3
> > > > > > > + #define UNICAM_PPM_PACK14 4
> > > > > > > + #define UNICAM_PPM_PACK16 5
> > > > > > > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > > > > > > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > > > > > > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > > > > > > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > > > > > > +
> > > > > > > +/* UNICAM_ICC Register */
> > > > > > > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > > > > > > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > > > > > > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > > > > > > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > > > > > > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > > > > > > +
> > > > > > > +/* UNICAM_DCS Register */
> > > > > > > +#define UNICAM_DIE BIT(0)
> > > > > > > +#define UNICAM_DIM BIT(1)
> > > > > > > +#define UNICAM_DBOB BIT(3)
> > > > > > > +#define UNICAM_FDE BIT(4)
> > > > > > > +#define UNICAM_LDP BIT(5)
> > > > > > > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > > > > > > +
> > > > > > > +/* UNICAM_DBCTL Register */
> > > > > > > +#define UNICAM_DBEN BIT(0)
> > > > > > > +#define UNICAM_BUF0_IE BIT(1)
> > > > > > > +#define UNICAM_BUF1_IE BIT(2)
> > > > > > > +
> > > > > > > +/* UNICAM_CMP[0,1] register */
> > > > > > > +#define UNICAM_PCE BIT(31)
> > > > > > > +#define UNICAM_GI BIT(9)
> > > > > > > +#define UNICAM_CPH BIT(8)
> > > > > > > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > > > > > > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > > > > > > +
> > > > > > > +/* UNICAM_MISC register */
> > > > > > > +#define UNICAM_FL0 BIT(6)
> > > > > > > +#define UNICAM_FL1 BIT(9)
> > > > > > > +
> > > > > > > +#endif
>
> --
> Regards,
>
> Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-09-01 8:41 ` Dave Stevenson
@ 2020-09-01 10:22 ` Jacopo Mondi
2020-09-01 16:37 ` Dave Stevenson
0 siblings, 1 reply; 111+ messages in thread
From: Jacopo Mondi @ 2020-09-01 10:22 UTC (permalink / raw)
To: Dave Stevenson, Sakari Ailus
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
Hi Dave,
+Sakari
On Tue, Sep 01, 2020 at 09:41:20AM +0100, Dave Stevenson wrote:
> Morning Laurent and Jacopo.
>
> On Mon, 31 Aug 2020 at 15:57, Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> >
> > Hi Jacopo,
> >
> > On Mon, Aug 31, 2020 at 04:46:37PM +0200, Jacopo Mondi wrote:
> > > On Mon, Aug 31, 2020 at 05:17:04PM +0300, Laurent Pinchart wrote:
> > > > On Mon, Aug 31, 2020 at 09:38:58AM +0200, Jacopo Mondi wrote:
> > > > > On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> > > > > > On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> > > > > > > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > > > > > > From: Naushir Patuck <naush@raspberrypi.com>
> > > > > > > >
> > > > > > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > > > > > Compared to the bcm2835-camera driver present in staging, this driver
> > > > > > > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > > > > > > the VC4 firmware running on the VPU.
> > > > > > > >
> > > > > > > > The commit is made up of a series of changes cherry-picked from the
> > > > > > > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > > > > > > additional enhancements, forward-ported to the mainline kernel.
> > > > > > > >
> > > > > > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > > > > > ---
> > > > > > > > Changes since v1:
> > > > > > > >
> > > > > > > > - Re-fetch mbus code from subdev on a g_fmt call
> > > > > > > > - Group all ioctl disabling together
> > > > > > > > - Fix reference counting in unicam_open
> > > > > > > > - Add support for VIDIOC_[S|G]_SELECTION
> > > > > > > > ---
> > > > > > >
> > > > > > > [snip]
> > > > > > >
> > > > > > > > +
> > > > > > > > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > > > > > > > +{
> > > > > > > > + struct platform_device *pdev = dev->pdev;
> > > > > > > > + struct v4l2_fwnode_endpoint ep = { 0 };
> > > > > > > > + struct device_node *ep_node;
> > > > > > > > + struct device_node *sensor_node;
> > > > > > > > + unsigned int lane;
> > > > > > > > + int ret = -EINVAL;
> > > > > > > > +
> > > > > > > > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > > > > > > > + &dev->max_data_lanes) < 0) {
> > > > > > > > + unicam_err(dev, "number of data lanes not set\n");
> > > > > > > > + return -EINVAL;
> > > > > > > > + }
> > > > > > >
> > > > > > > mmm, this seems like a good use case for the newly introduced
> > > > > > > get_mbus_config pad operation, isn't it ?
> > > > > > >
> > > > > > > We could drop the custom "brcm,num-data-lanes" property, use the
> > > > > > > standard "num-lanes" to describe the number of wired data lanes, and
> > > > > > > get the number of actually used ones at s_stream(1) time with the new pad
> > > > > > > operation
> > > > > > >
> > > > > > > What do you think ? This would require implemeting the operation in
> > > > > > > each subdev the UNICAM interface has to work with, but this doesn't
> > > > > > > sound too much of a work.
> > > > > > >
> > > > > > > For reference this is how it happens on rcar+adv748x:
> > > > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> > > > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
> > > > > >
> > > > > > Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> > > > > > The former tells how many data lanes the unicam IP has (the same IP,
> > > > > > with the same compatible string, is instantiated in different SoCs with
> > > > > > different number of lanes), while the latter queries at runtime how many
> > > > > > data lanes the remote subdev will use for its current configuration. The
> > > > >
> > > > > Can't we get the number of available data lanes just by parsing the
> > > > > endpoint ?
> > > >
> > > > We can get the number of data lanes routed on the board, which may not
> > > > match the number of data lanes available in the IP. I'll let Dave
> > > > comment on this, I recall we had a lengthy discussion on this topic in
> > > > the past, but I don't remember the details :-S
>
> Thread "[PATCH 10/10] media: bcm2835-unicam: Simplify OF parsing"
> posted to the private libcamera list Sun, 19 Apr. I can forward on if
> others have deleted it.
>
Thanks. I've now read the thread again.
I think my disagreement is with the fact that data-lanes shall
represent the number of data lanes connected between the sensor and the
receiver.
data-lanes, to me, represents the number (and index) of the lanes
instantiated on the silicon, or even exposed on the connector. In
plain English: "the number of available data lanes".
The sensor will describe it's own lanes (which might be less than the
ones available in the receiver) and at streaming time the receiver
gets how many of them shall be used by using get_mbus_config(). This
would basically move the notion of "how many lanes are in use" to
run-time, leaving the HW description of "what's available in the HW"
to DT. It seems a much neater solution to me, also considering the
current implementation does not scale if we throw muxes and pluggable
devices in the mix.
This isn't a discussion on this patch only, and I know Laurent and
Sakari already had gone through that multiple times. I see if I have
energies to fight that battle, but for this series let's live with the
custom property.
> > > I think the max number of number of lanes in the IP should come in DT
> > > by restricting the number of accepted data-lanes value depending on
> > > the compatible string.
> >
> > Unless I'm mistaken we have a single compatibility string.
>
> Currently we have a single compatible string.
> So you want a "brcm,bcm2835-unicam-2-lane" and "brcm,bcm2835-unicam-4-lane"?
No please :) I assumed you had different ones.
> You can't guarantee that unicam0 is always 2 lane and unicam1 is
> always 4 lane forever more - I know there are Broadcom chips with a
> unicam2 for a start.
> I defer to the DT gatekeepers, but that feels ugly.
>
> > > > > I'm aware of the implication that the 'data-lanes' property
> > > > > would differ in the bridge and sensor endpoints, but now that we have
> > > > > a mechanism to get that information wihtout parsing the remote
> > > > > endpoint it doesn't seem wrong to me.
> > > >
> > > > That's not the issue at hand. The data-lanes property need to match on
> > > > both sides of a link, as it tells how many data lanes are routed on the
> > > > board. Of course that becomes problematic when we start using overlays
> > > > for pluggable boards, as the two sides can be designed with a different
> > > > number of routed lanes.
> > >
> > > It actually is. As I read the current implementation, UNICAM has 4
> > > lanes available, 2 are enabled in the endpoint, and the remote uses 2.
> >
> > That's because the board has 2 data lanes routed.
> >
> > > If we set the number of lanes in the endpoint to 4 (the actually
> > > available ones) and use only the two required by querying the remote
> > > end at s_stream time, we don't need any custom sourcery
> >
> > That would be incorrect, as there are two data lanes routed, so you
> > shouldn't set the data-lanes property to 4 lanes in the endpoint.
> >
> > > > > And that's not different from what it's done in R-Car (half-true: we
> > > > > have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
> > > > > as you said the number of active data lanes can be downscaled to
> > > > > transmit low resolution streams).
> > > > >
> > > > > I think Dave already had this conversation in the past, I might dig
> > > > > the archive and search for a reference.
>
> Referenced above.
>
> > > > > > DT property serves as a sanity check, to ensure to no more than the
> > > > > > number of supported data lanes is used, while the subdev operation
> > > > > > should be used in the unicam driver to support sources that will
> > > > > > transmit over a different number of data lanes depending on the
> > > > > > configuration (mostly depending on the resolution). We don't have to
> > > > >
> > > > > I think reading the remote's mbus configuration might also help in
> > > > > support connecting pluggable devices with a different number of wired
> > > > > data lanes than the bridge's available ones.
> > > >
> > > > No, that we shouldn't do. DT properties need to be interpreted within
> > > > the context of a compatible string, reading properties of a remote node
> > > > is a no-go in most cases. It's really really something that must not be
> > > > done without careful consideration, and is never a simple way to work
> > > > around a problem.
> > >
> > > What I mean is that get_mbus_config() allows you to retrieve those
> > > information -without- having to read the remote endpoint, which has
> > > always been a blocker for this kind of operations (and I agree it
> > > shouldn't be done).
>
> I like get_mbus_config as far as it goes.
> When it last came up I mentioned that it doesn't convey things like
> the continuous clock setting, so you have to have the DT endpoints
> configured identically for that to avoid issues, and if you start to
> involve CSI2 mux chips then both sources have to have the same
> requirements.
The merged version of get_mbus_config supports describing
non-continuous and continuous clock, as we're currently re-using the
same V4L2_MBUS_ flags the legacy g|s_mbus_config operations used.
> Personally I'd love to see get_mbus_config being the correct way of
> configuring all receiver parameters as you'd then be guaranteed to
> have source and receiver matching. Drop back to the receiver's DT
> endpoint when get_mbus_config isn't supported.
>
As said, I tend to agree, even if it's a bit freagile having receivers
and sources agree to -what- has to come from get_mbus_config and from
DT. But moving forward that would be my preference indeed.
> > > > > Am I missing something maybe? The non-connected lanes on the bride
> > > > > will simply stay unused, can't they ?
> > > > >
> > > > > > implement support for get_mbus_config just yet, it can be delayed until
> > > > > > we have a source that requires it.
> > > > >
> > > > > I sketched a few patches yesterday basically appling what's done in
> > > > > 5cef0785362ed and a61b1b5d41735. Is a custom property better in your
> > > > > opinion ?
> > > >
> > > > Again, the custom property only specifies the number of data lanes
> > > > implemented in the unicam IP core. It's nothing else than that, and does
> > > > *not* convey any information about how many data lanes the sensor will
> > > > use.
> > >
> > > It shouldn't, and to me that information should come from DT as we do
> > > for all other platforms.
> > >
> > > Let's see, if feels a bit a waste to use a custom property for
> > > something we can do without, and usage of get_mbus_config() would
> > > allow to support switching devices at run-time with a different number
> > > of data-lanes, as use case I know Dave has downstream.
> >
> > This is *still* not what brcm,num-lanes is about... get_mbus_config()
> > will not tell you how many data lanes the unicam IP supports.
>
> Use case of a TC358743 HDMI to CSI2 bridge connected to Unicam0 (2 lane).
> TC358743 supports get_mbus_config and says it wants 4 lanes for 1080p60 RGB888.
> Unicam calls get_mbus_config and gets told 4.
> So how do we detect and reject that invalid configuration? It can't
unicam0 will have data-lanes=<1 2> and such it's max number of data
lanes is 2, and if you get 4 from get_mbus_config() you refuse that.
I now wonder about lane reordering, we can express it in DT but not
with V4L2_MBUS_* flags. One more reason to supersed these flags with
more expressive fields, even if lane re-ordering seems to be a corner
case really.
Anyway, I'm a bit sad of upstreaming this with a custom property, but
this is a broader discussion and I don't want it to block unicam
acceptance.
> rely on the Unicam data-lanes as that is supposed to reflect the
> number of active lanes, but it's not something that can be otherwise
> detected and blocked. Start streaming and it'll just sit there dumbly
> doing nothing.
>
> Yes we did discuss that most of the Pi boards have only 2 of the 4
> data lanes from Unicam1 wired out to the camera connector, so whilst
> technically not what the binding says, it is possible to then drop
> num-data-lanes down to 2 (though the IP has 4 lanes) and get
> validation of the config in that case as well.
>
> Dave
>
> > > > > > > +
> > > > > > > > + /* Get the local endpoint and remote device. */
> > > > > > > > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > > > > > > > + if (!ep_node) {
> > > > > > > > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > > > > > > > + return -EINVAL;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > > > > > > > +
> > > > > > > > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > > > > > > > + if (!sensor_node) {
> > > > > > > > + unicam_dbg(3, dev, "can't get remote parent\n");
> > > > > > > > + goto cleanup_exit;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > > > > > > > +
> > > > > > > > + /* Parse the local endpoint and validate its configuration. */
> > > > > > > > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > > > > > > > +
> > > > > > > > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > > > > > > > + ep.bus_type);
> > > > > > > > +
> > > > > > > > + dev->bus_type = ep.bus_type;
> > > > > > > > +
> > > > > > > > + switch (ep.bus_type) {
> > > > > > > > + case V4L2_MBUS_CSI2_DPHY:
> > > > > > > > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > > > > > > > + case 1:
> > > > > > > > + case 2:
> > > > > > > > + case 4:
> > > > > > > > + break;
> > > > > > > > +
> > > > > > > > + default:
> > > > > > > > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > > > > > > > + sensor_node,
> > > > > > > > + ep.bus.mipi_csi2.num_data_lanes);
> > > > > > > > + goto cleanup_exit;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > > > > > > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > > > > > > > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > > > > > > > + sensor_node);
> > > > > > > > + goto cleanup_exit;
> > > > > > > > + }
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > > > > > > > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > > > > > > > + ep.bus.mipi_csi2.num_data_lanes,
> > > > > > > > + dev->max_data_lanes);
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > > > > > > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > > > > > > > +
> > > > > > > > + break;
> > > > > > > > +
> > > > > > > > + case V4L2_MBUS_CCP2:
> > > > > > > > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > > > > > > > + ep.bus.mipi_csi1.data_lane != 1) {
> > > > > > > > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > > > > > > > + sensor_node);
> > > > > > > > + goto cleanup_exit;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + dev->max_data_lanes = 1;
> > > > > > > > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > > > > > > > + break;
> > > > > > > > +
> > > > > > > > + default:
> > > > > > > > + /* Unsupported bus type */
> > > > > > > > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > > > > > > > + sensor_node, ep.bus_type);
> > > > > > > > + goto cleanup_exit;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > > > > > > > + sensor_node,
> > > > > > > > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > > > > > > > + dev->max_data_lanes, dev->bus_flags);
> > > > > > > > +
> > > > > > > > + /* Initialize and register the async notifier. */
> > > > > > > > + v4l2_async_notifier_init(&dev->notifier);
> > > > > > > > + dev->notifier.ops = &unicam_async_ops;
> > > > > > > > +
> > > > > > > > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > > > > > > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > > > > > > > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > > > > > > > + if (ret) {
> > > > > > > > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > > > > > > > + goto cleanup_exit;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > > > > > > > + if (ret) {
> > > > > > > > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > > > > > > > + ret = -EINVAL;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > +cleanup_exit:
> > > > > > > > + of_node_put(sensor_node);
> > > > > > > > + of_node_put(ep_node);
> > > > > > > > +
> > > > > > > > + return ret;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static int unicam_probe(struct platform_device *pdev)
> > > > > > > > +{
> > > > > > > > + struct unicam_device *unicam;
> > > > > > > > + int ret;
> > > > > > > > +
> > > > > > > > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > > > > > > > + if (!unicam)
> > > > > > > > + return -ENOMEM;
> > > > > > > > +
> > > > > > > > + kref_init(&unicam->kref);
> > > > > > > > + unicam->pdev = pdev;
> > > > > > > > +
> > > > > > > > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > > > > > > > + if (IS_ERR(unicam->base)) {
> > > > > > > > + unicam_err(unicam, "Failed to get main io block\n");
> > > > > > > > + ret = PTR_ERR(unicam->base);
> > > > > > > > + goto err_unicam_put;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > > > > > > > + if (IS_ERR(unicam->clk_gate_base)) {
> > > > > > > > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > > > > > > > + ret = PTR_ERR(unicam->clk_gate_base);
> > > > > > > > + goto err_unicam_put;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > > > > > > > + if (IS_ERR(unicam->clock)) {
> > > > > > > > + unicam_err(unicam, "Failed to get clock\n");
> > > > > > > > + ret = PTR_ERR(unicam->clock);
> > > > > > > > + goto err_unicam_put;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + ret = platform_get_irq(pdev, 0);
> > > > > > > > + if (ret <= 0) {
> > > > > > > > + dev_err(&pdev->dev, "No IRQ resource\n");
> > > > > > > > + ret = -EINVAL;
> > > > > > > > + goto err_unicam_put;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > > > > > > > + "unicam_capture0", unicam);
> > > > > > > > + if (ret) {
> > > > > > > > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > > > > > > > + ret = -EINVAL;
> > > > > > > > + goto err_unicam_put;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + unicam->mdev.dev = &pdev->dev;
> > > > > > > > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > > > > > > + sizeof(unicam->mdev.model));
> > > > > > > > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > > > > > > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > > > > > > > + "platform:%s", dev_name(&pdev->dev));
> > > > > > > > + unicam->mdev.hw_revision = 0;
> > > > > > > > +
> > > > > > > > + media_device_init(&unicam->mdev);
> > > > > > > > +
> > > > > > > > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > > > > > > > +
> > > > > > > > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > > > > > > > + if (ret) {
> > > > > > > > + unicam_err(unicam,
> > > > > > > > + "Unable to register v4l2 device.\n");
> > > > > > > > + goto err_unicam_put;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + ret = media_device_register(&unicam->mdev);
> > > > > > > > + if (ret < 0) {
> > > > > > > > + unicam_err(unicam,
> > > > > > > > + "Unable to register media-controller device.\n");
> > > > > > > > + goto err_v4l2_unregister;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + /* Reserve space for the controls */
> > > > > > > > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > > > > > > > + if (ret < 0)
> > > > > > > > + goto err_media_unregister;
> > > > > > > > +
> > > > > > > > + /* set the driver data in platform device */
> > > > > > > > + platform_set_drvdata(pdev, unicam);
> > > > > > > > +
> > > > > > > > + ret = of_unicam_connect_subdevs(unicam);
> > > > > > > > + if (ret) {
> > > > > > > > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > > > > > > > + goto err_media_unregister;
> > > > > > > > + }
> > > > > > > > +
> > > > > > > > + /* Enable the block power domain */
> > > > > > > > + pm_runtime_enable(&pdev->dev);
> > > > > > > > +
> > > > > > > > + return 0;
> > > > > > > > +
> > > > > > > > +err_media_unregister:
> > > > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > > > +err_v4l2_unregister:
> > > > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > > > +err_unicam_put:
> > > > > > > > + unicam_put(unicam);
> > > > > > > > +
> > > > > > > > + return ret;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static int unicam_remove(struct platform_device *pdev)
> > > > > > > > +{
> > > > > > > > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > > > > > > > +
> > > > > > > > + unicam_dbg(2, unicam, "%s\n", __func__);
> > > > > > > > +
> > > > > > > > + v4l2_async_notifier_unregister(&unicam->notifier);
> > > > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > > > + unregister_nodes(unicam);
> > > > > > > > +
> > > > > > > > + pm_runtime_disable(&pdev->dev);
> > > > > > > > +
> > > > > > > > + return 0;
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +static const struct of_device_id unicam_of_match[] = {
> > > > > > > > + { .compatible = "brcm,bcm2835-unicam", },
> > > > > > > > + { /* sentinel */ },
> > > > > > > > +};
> > > > > > > > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > > > > > > > +
> > > > > > > > +static struct platform_driver unicam_driver = {
> > > > > > > > + .probe = unicam_probe,
> > > > > > > > + .remove = unicam_remove,
> > > > > > > > + .driver = {
> > > > > > > > + .name = UNICAM_MODULE_NAME,
> > > > > > > > + .of_match_table = of_match_ptr(unicam_of_match),
> > > > > > > > + },
> > > > > > > > +};
> > > > > > > > +
> > > > > > > > +module_platform_driver(unicam_driver);
> > > > > > > > +
> > > > > > > > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > > > > > > > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > > > > > > > +MODULE_LICENSE("GPL");
> > > > > > > > +MODULE_VERSION(UNICAM_VERSION);
> > > > > > > > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > > > new file mode 100644
> > > > > > > > index 000000000000..ae059a171d0f
> > > > > > > > --- /dev/null
> > > > > > > > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > > > @@ -0,0 +1,253 @@
> > > > > > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > > > > > +
> > > > > > > > +/*
> > > > > > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > > > > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > > > + */
> > > > > > > > +
> > > > > > > > +#ifndef VC4_REGS_UNICAM_H
> > > > > > > > +#define VC4_REGS_UNICAM_H
> > > > > > > > +
> > > > > > > > +/*
> > > > > > > > + * The following values are taken from files found within the code drop
> > > > > > > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > > > > > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > > > > > > + * They have been modified to be only the register offset.
> > > > > > > > + */
> > > > > > > > +#define UNICAM_CTRL 0x000
> > > > > > > > +#define UNICAM_STA 0x004
> > > > > > > > +#define UNICAM_ANA 0x008
> > > > > > > > +#define UNICAM_PRI 0x00c
> > > > > > > > +#define UNICAM_CLK 0x010
> > > > > > > > +#define UNICAM_CLT 0x014
> > > > > > > > +#define UNICAM_DAT0 0x018
> > > > > > > > +#define UNICAM_DAT1 0x01c
> > > > > > > > +#define UNICAM_DAT2 0x020
> > > > > > > > +#define UNICAM_DAT3 0x024
> > > > > > > > +#define UNICAM_DLT 0x028
> > > > > > > > +#define UNICAM_CMP0 0x02c
> > > > > > > > +#define UNICAM_CMP1 0x030
> > > > > > > > +#define UNICAM_CAP0 0x034
> > > > > > > > +#define UNICAM_CAP1 0x038
> > > > > > > > +#define UNICAM_ICTL 0x100
> > > > > > > > +#define UNICAM_ISTA 0x104
> > > > > > > > +#define UNICAM_IDI0 0x108
> > > > > > > > +#define UNICAM_IPIPE 0x10c
> > > > > > > > +#define UNICAM_IBSA0 0x110
> > > > > > > > +#define UNICAM_IBEA0 0x114
> > > > > > > > +#define UNICAM_IBLS 0x118
> > > > > > > > +#define UNICAM_IBWP 0x11c
> > > > > > > > +#define UNICAM_IHWIN 0x120
> > > > > > > > +#define UNICAM_IHSTA 0x124
> > > > > > > > +#define UNICAM_IVWIN 0x128
> > > > > > > > +#define UNICAM_IVSTA 0x12c
> > > > > > > > +#define UNICAM_ICC 0x130
> > > > > > > > +#define UNICAM_ICS 0x134
> > > > > > > > +#define UNICAM_IDC 0x138
> > > > > > > > +#define UNICAM_IDPO 0x13c
> > > > > > > > +#define UNICAM_IDCA 0x140
> > > > > > > > +#define UNICAM_IDCD 0x144
> > > > > > > > +#define UNICAM_IDS 0x148
> > > > > > > > +#define UNICAM_DCS 0x200
> > > > > > > > +#define UNICAM_DBSA0 0x204
> > > > > > > > +#define UNICAM_DBEA0 0x208
> > > > > > > > +#define UNICAM_DBWP 0x20c
> > > > > > > > +#define UNICAM_DBCTL 0x300
> > > > > > > > +#define UNICAM_IBSA1 0x304
> > > > > > > > +#define UNICAM_IBEA1 0x308
> > > > > > > > +#define UNICAM_IDI1 0x30c
> > > > > > > > +#define UNICAM_DBSA1 0x310
> > > > > > > > +#define UNICAM_DBEA1 0x314
> > > > > > > > +#define UNICAM_MISC 0x400
> > > > > > > > +
> > > > > > > > +/*
> > > > > > > > + * The following bitmasks are from the kernel released by Broadcom
> > > > > > > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > > > > > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > > > > > > + * Unicam block as BCM2835, as defined in eg
> > > > > > > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > > > > > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > > > > > > + *
> > > > > > > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > > > > > > + */
> > > > > > > > +/* UNICAM_CTRL Register */
> > > > > > > > +#define UNICAM_CPE BIT(0)
> > > > > > > > +#define UNICAM_MEM BIT(1)
> > > > > > > > +#define UNICAM_CPR BIT(2)
> > > > > > > > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > > > > > > > +#define UNICAM_CPM_CSI2 0
> > > > > > > > +#define UNICAM_CPM_CCP2 1
> > > > > > > > +#define UNICAM_SOE BIT(4)
> > > > > > > > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > > > > > > > +#define UNICAM_DCM_STROBE 0
> > > > > > > > +#define UNICAM_DCM_DATA 1
> > > > > > > > +#define UNICAM_SLS BIT(6)
> > > > > > > > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > > > > > > > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > > > > > > > +
> > > > > > > > +/* UNICAM_STA Register */
> > > > > > > > +#define UNICAM_SYN BIT(0)
> > > > > > > > +#define UNICAM_CS BIT(1)
> > > > > > > > +#define UNICAM_SBE BIT(2)
> > > > > > > > +#define UNICAM_PBE BIT(3)
> > > > > > > > +#define UNICAM_HOE BIT(4)
> > > > > > > > +#define UNICAM_PLE BIT(5)
> > > > > > > > +#define UNICAM_SSC BIT(6)
> > > > > > > > +#define UNICAM_CRCE BIT(7)
> > > > > > > > +#define UNICAM_OES BIT(8)
> > > > > > > > +#define UNICAM_IFO BIT(9)
> > > > > > > > +#define UNICAM_OFO BIT(10)
> > > > > > > > +#define UNICAM_BFO BIT(11)
> > > > > > > > +#define UNICAM_DL BIT(12)
> > > > > > > > +#define UNICAM_PS BIT(13)
> > > > > > > > +#define UNICAM_IS BIT(14)
> > > > > > > > +#define UNICAM_PI0 BIT(15)
> > > > > > > > +#define UNICAM_PI1 BIT(16)
> > > > > > > > +#define UNICAM_FSI_S BIT(17)
> > > > > > > > +#define UNICAM_FEI_S BIT(18)
> > > > > > > > +#define UNICAM_LCI_S BIT(19)
> > > > > > > > +#define UNICAM_BUF0_RDY BIT(20)
> > > > > > > > +#define UNICAM_BUF0_NO BIT(21)
> > > > > > > > +#define UNICAM_BUF1_RDY BIT(22)
> > > > > > > > +#define UNICAM_BUF1_NO BIT(23)
> > > > > > > > +#define UNICAM_DI BIT(24)
> > > > > > > > +
> > > > > > > > +#define UNICAM_STA_MASK_ALL \
> > > > > > > > + (UNICAM_DL + \
> > > > > > > > + UNICAM_SBE + \
> > > > > > > > + UNICAM_PBE + \
> > > > > > > > + UNICAM_HOE + \
> > > > > > > > + UNICAM_PLE + \
> > > > > > > > + UNICAM_SSC + \
> > > > > > > > + UNICAM_CRCE + \
> > > > > > > > + UNICAM_IFO + \
> > > > > > > > + UNICAM_OFO + \
> > > > > > > > + UNICAM_PS + \
> > > > > > > > + UNICAM_PI0 + \
> > > > > > > > + UNICAM_PI1)
> > > > > > > > +
> > > > > > > > +/* UNICAM_ANA Register */
> > > > > > > > +#define UNICAM_APD BIT(0)
> > > > > > > > +#define UNICAM_BPD BIT(1)
> > > > > > > > +#define UNICAM_AR BIT(2)
> > > > > > > > +#define UNICAM_DDL BIT(3)
> > > > > > > > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > > > > > > > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > > > > > > > +
> > > > > > > > +/* UNICAM_PRI Register */
> > > > > > > > +#define UNICAM_PE BIT(0)
> > > > > > > > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > > > > > > > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > > > > > > > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > > > > > > > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > > > > > > > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > > > > > > > +
> > > > > > > > +/* UNICAM_CLK Register */
> > > > > > > > +#define UNICAM_CLE BIT(0)
> > > > > > > > +#define UNICAM_CLPD BIT(1)
> > > > > > > > +#define UNICAM_CLLPE BIT(2)
> > > > > > > > +#define UNICAM_CLHSE BIT(3)
> > > > > > > > +#define UNICAM_CLTRE BIT(4)
> > > > > > > > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > > > > > > > +#define UNICAM_CLSTE BIT(29)
> > > > > > > > +
> > > > > > > > +/* UNICAM_CLT Register */
> > > > > > > > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > > > > > > > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > > > > > > > +
> > > > > > > > +/* UNICAM_DATn Registers */
> > > > > > > > +#define UNICAM_DLE BIT(0)
> > > > > > > > +#define UNICAM_DLPD BIT(1)
> > > > > > > > +#define UNICAM_DLLPE BIT(2)
> > > > > > > > +#define UNICAM_DLHSE BIT(3)
> > > > > > > > +#define UNICAM_DLTRE BIT(4)
> > > > > > > > +#define UNICAM_DLSM BIT(5)
> > > > > > > > +#define UNICAM_DLFO BIT(28)
> > > > > > > > +#define UNICAM_DLSTE BIT(29)
> > > > > > > > +
> > > > > > > > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > > > > > > > +
> > > > > > > > +/* UNICAM_DLT Register */
> > > > > > > > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > > > > > > > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > > > > > > > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > > > > > > > +
> > > > > > > > +/* UNICAM_ICTL Register */
> > > > > > > > +#define UNICAM_FSIE BIT(0)
> > > > > > > > +#define UNICAM_FEIE BIT(1)
> > > > > > > > +#define UNICAM_IBOB BIT(2)
> > > > > > > > +#define UNICAM_FCM BIT(3)
> > > > > > > > +#define UNICAM_TFC BIT(4)
> > > > > > > > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > > > > > > > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > > > > > > > +
> > > > > > > > +/* UNICAM_IDI0/1 Register */
> > > > > > > > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > > > > > > > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > > > > > > > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > > > > > > > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > > > > > > > +
> > > > > > > > +/* UNICAM_ISTA Register */
> > > > > > > > +#define UNICAM_FSI BIT(0)
> > > > > > > > +#define UNICAM_FEI BIT(1)
> > > > > > > > +#define UNICAM_LCI BIT(2)
> > > > > > > > +
> > > > > > > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > > > > > > > +
> > > > > > > > +/* UNICAM_IPIPE Register */
> > > > > > > > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > > > > > > > + /* Unpacking modes */
> > > > > > > > + #define UNICAM_PUM_NONE 0
> > > > > > > > + #define UNICAM_PUM_UNPACK6 1
> > > > > > > > + #define UNICAM_PUM_UNPACK7 2
> > > > > > > > + #define UNICAM_PUM_UNPACK8 3
> > > > > > > > + #define UNICAM_PUM_UNPACK10 4
> > > > > > > > + #define UNICAM_PUM_UNPACK12 5
> > > > > > > > + #define UNICAM_PUM_UNPACK14 6
> > > > > > > > + #define UNICAM_PUM_UNPACK16 7
> > > > > > > > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > > > > > > > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > > > > > > > + /* Packing modes */
> > > > > > > > + #define UNICAM_PPM_NONE 0
> > > > > > > > + #define UNICAM_PPM_PACK8 1
> > > > > > > > + #define UNICAM_PPM_PACK10 2
> > > > > > > > + #define UNICAM_PPM_PACK12 3
> > > > > > > > + #define UNICAM_PPM_PACK14 4
> > > > > > > > + #define UNICAM_PPM_PACK16 5
> > > > > > > > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > > > > > > > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > > > > > > > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > > > > > > > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > > > > > > > +
> > > > > > > > +/* UNICAM_ICC Register */
> > > > > > > > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > > > > > > > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > > > > > > > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > > > > > > > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > > > > > > > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > > > > > > > +
> > > > > > > > +/* UNICAM_DCS Register */
> > > > > > > > +#define UNICAM_DIE BIT(0)
> > > > > > > > +#define UNICAM_DIM BIT(1)
> > > > > > > > +#define UNICAM_DBOB BIT(3)
> > > > > > > > +#define UNICAM_FDE BIT(4)
> > > > > > > > +#define UNICAM_LDP BIT(5)
> > > > > > > > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > > > > > > > +
> > > > > > > > +/* UNICAM_DBCTL Register */
> > > > > > > > +#define UNICAM_DBEN BIT(0)
> > > > > > > > +#define UNICAM_BUF0_IE BIT(1)
> > > > > > > > +#define UNICAM_BUF1_IE BIT(2)
> > > > > > > > +
> > > > > > > > +/* UNICAM_CMP[0,1] register */
> > > > > > > > +#define UNICAM_PCE BIT(31)
> > > > > > > > +#define UNICAM_GI BIT(9)
> > > > > > > > +#define UNICAM_CPH BIT(8)
> > > > > > > > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > > > > > > > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > > > > > > > +
> > > > > > > > +/* UNICAM_MISC register */
> > > > > > > > +#define UNICAM_FL0 BIT(6)
> > > > > > > > +#define UNICAM_FL1 BIT(9)
> > > > > > > > +
> > > > > > > > +#endif
> >
> > --
> > Regards,
> >
> > Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-09-01 10:22 ` Jacopo Mondi
@ 2020-09-01 16:37 ` Dave Stevenson
2020-09-01 17:11 ` Laurent Pinchart
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-09-01 16:37 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Sakari Ailus, Laurent Pinchart, Linux Media Mailing List,
Kieran Bingham, Niklas Söderlund, Naushir Patuck
Hi Jacopo.
On Tue, 1 Sep 2020 at 11:19, Jacopo Mondi <jacopo@jmondi.org> wrote:
>
> Hi Dave,
> +Sakari
>
> On Tue, Sep 01, 2020 at 09:41:20AM +0100, Dave Stevenson wrote:
> > Morning Laurent and Jacopo.
> >
> > On Mon, 31 Aug 2020 at 15:57, Laurent Pinchart
> > <laurent.pinchart@ideasonboard.com> wrote:
> > >
> > > Hi Jacopo,
> > >
> > > On Mon, Aug 31, 2020 at 04:46:37PM +0200, Jacopo Mondi wrote:
> > > > On Mon, Aug 31, 2020 at 05:17:04PM +0300, Laurent Pinchart wrote:
> > > > > On Mon, Aug 31, 2020 at 09:38:58AM +0200, Jacopo Mondi wrote:
> > > > > > On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> > > > > > > On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> > > > > > > > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > > > > > > > From: Naushir Patuck <naush@raspberrypi.com>
> > > > > > > > >
> > > > > > > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > > > > > > Compared to the bcm2835-camera driver present in staging, this driver
> > > > > > > > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > > > > > > > the VC4 firmware running on the VPU.
> > > > > > > > >
> > > > > > > > > The commit is made up of a series of changes cherry-picked from the
> > > > > > > > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > > > > > > > additional enhancements, forward-ported to the mainline kernel.
> > > > > > > > >
> > > > > > > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > > > > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > > > > > > ---
> > > > > > > > > Changes since v1:
> > > > > > > > >
> > > > > > > > > - Re-fetch mbus code from subdev on a g_fmt call
> > > > > > > > > - Group all ioctl disabling together
> > > > > > > > > - Fix reference counting in unicam_open
> > > > > > > > > - Add support for VIDIOC_[S|G]_SELECTION
> > > > > > > > > ---
> > > > > > > >
> > > > > > > > [snip]
> > > > > > > >
> > > > > > > > > +
> > > > > > > > > +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> > > > > > > > > +{
> > > > > > > > > + struct platform_device *pdev = dev->pdev;
> > > > > > > > > + struct v4l2_fwnode_endpoint ep = { 0 };
> > > > > > > > > + struct device_node *ep_node;
> > > > > > > > > + struct device_node *sensor_node;
> > > > > > > > > + unsigned int lane;
> > > > > > > > > + int ret = -EINVAL;
> > > > > > > > > +
> > > > > > > > > + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> > > > > > > > > + &dev->max_data_lanes) < 0) {
> > > > > > > > > + unicam_err(dev, "number of data lanes not set\n");
> > > > > > > > > + return -EINVAL;
> > > > > > > > > + }
> > > > > > > >
> > > > > > > > mmm, this seems like a good use case for the newly introduced
> > > > > > > > get_mbus_config pad operation, isn't it ?
> > > > > > > >
> > > > > > > > We could drop the custom "brcm,num-data-lanes" property, use the
> > > > > > > > standard "num-lanes" to describe the number of wired data lanes, and
> > > > > > > > get the number of actually used ones at s_stream(1) time with the new pad
> > > > > > > > operation
> > > > > > > >
> > > > > > > > What do you think ? This would require implemeting the operation in
> > > > > > > > each subdev the UNICAM interface has to work with, but this doesn't
> > > > > > > > sound too much of a work.
> > > > > > > >
> > > > > > > > For reference this is how it happens on rcar+adv748x:
> > > > > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> > > > > > > > https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
> > > > > > >
> > > > > > > Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> > > > > > > The former tells how many data lanes the unicam IP has (the same IP,
> > > > > > > with the same compatible string, is instantiated in different SoCs with
> > > > > > > different number of lanes), while the latter queries at runtime how many
> > > > > > > data lanes the remote subdev will use for its current configuration. The
> > > > > >
> > > > > > Can't we get the number of available data lanes just by parsing the
> > > > > > endpoint ?
> > > > >
> > > > > We can get the number of data lanes routed on the board, which may not
> > > > > match the number of data lanes available in the IP. I'll let Dave
> > > > > comment on this, I recall we had a lengthy discussion on this topic in
> > > > > the past, but I don't remember the details :-S
> >
> > Thread "[PATCH 10/10] media: bcm2835-unicam: Simplify OF parsing"
> > posted to the private libcamera list Sun, 19 Apr. I can forward on if
> > others have deleted it.
> >
>
> Thanks. I've now read the thread again.
>
> I think my disagreement is with the fact that data-lanes shall
> represent the number of data lanes connected between the sensor and the
> receiver.
>
> data-lanes, to me, represents the number (and index) of the lanes
> instantiated on the silicon, or even exposed on the connector. In
> plain English: "the number of available data lanes".
That would be my wish too, but it's not the current usage.
> The sensor will describe it's own lanes (which might be less than the
> ones available in the receiver) and at streaming time the receiver
> gets how many of them shall be used by using get_mbus_config(). This
> would basically move the notion of "how many lanes are in use" to
> run-time, leaving the HW description of "what's available in the HW"
> to DT. It seems a much neater solution to me, also considering the
> current implementation does not scale if we throw muxes and pluggable
> devices in the mix.
Yes, if get_mbus_config() was mandatory for all sensor/source drivers
to implement then that would work, but currently I think there are
only about 4 drivers that implement it.
It'd be nice if there was a way for the framework to take care of it
for the simple drivers that purely take the config from DT, but I'm
not sure that's feasible.
> This isn't a discussion on this patch only, and I know Laurent and
> Sakari already had gone through that multiple times. I see if I have
> energies to fight that battle, but for this series let's live with the
> custom property.
>
> > > > I think the max number of number of lanes in the IP should come in DT
> > > > by restricting the number of accepted data-lanes value depending on
> > > > the compatible string.
> > >
> > > Unless I'm mistaken we have a single compatibility string.
> >
> > Currently we have a single compatible string.
> > So you want a "brcm,bcm2835-unicam-2-lane" and "brcm,bcm2835-unicam-4-lane"?
>
> No please :) I assumed you had different ones.
I'm glad you said that :-)
Other than the number of lanes the IP in all the variations of Pi is
identical, so adding multiple compatibles seemed excessive. It's the
same for a fair number of the IP blocks in the SoC such as I2C and
SPI.
> > You can't guarantee that unicam0 is always 2 lane and unicam1 is
> > always 4 lane forever more - I know there are Broadcom chips with a
> > unicam2 for a start.
> > I defer to the DT gatekeepers, but that feels ugly.
> >
> > > > > > I'm aware of the implication that the 'data-lanes' property
> > > > > > would differ in the bridge and sensor endpoints, but now that we have
> > > > > > a mechanism to get that information wihtout parsing the remote
> > > > > > endpoint it doesn't seem wrong to me.
> > > > >
> > > > > That's not the issue at hand. The data-lanes property need to match on
> > > > > both sides of a link, as it tells how many data lanes are routed on the
> > > > > board. Of course that becomes problematic when we start using overlays
> > > > > for pluggable boards, as the two sides can be designed with a different
> > > > > number of routed lanes.
> > > >
> > > > It actually is. As I read the current implementation, UNICAM has 4
> > > > lanes available, 2 are enabled in the endpoint, and the remote uses 2.
> > >
> > > That's because the board has 2 data lanes routed.
> > >
> > > > If we set the number of lanes in the endpoint to 4 (the actually
> > > > available ones) and use only the two required by querying the remote
> > > > end at s_stream time, we don't need any custom sourcery
> > >
> > > That would be incorrect, as there are two data lanes routed, so you
> > > shouldn't set the data-lanes property to 4 lanes in the endpoint.
> > >
> > > > > > And that's not different from what it's done in R-Car (half-true: we
> > > > > > have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
> > > > > > as you said the number of active data lanes can be downscaled to
> > > > > > transmit low resolution streams).
> > > > > >
> > > > > > I think Dave already had this conversation in the past, I might dig
> > > > > > the archive and search for a reference.
> >
> > Referenced above.
> >
> > > > > > > DT property serves as a sanity check, to ensure to no more than the
> > > > > > > number of supported data lanes is used, while the subdev operation
> > > > > > > should be used in the unicam driver to support sources that will
> > > > > > > transmit over a different number of data lanes depending on the
> > > > > > > configuration (mostly depending on the resolution). We don't have to
> > > > > >
> > > > > > I think reading the remote's mbus configuration might also help in
> > > > > > support connecting pluggable devices with a different number of wired
> > > > > > data lanes than the bridge's available ones.
> > > > >
> > > > > No, that we shouldn't do. DT properties need to be interpreted within
> > > > > the context of a compatible string, reading properties of a remote node
> > > > > is a no-go in most cases. It's really really something that must not be
> > > > > done without careful consideration, and is never a simple way to work
> > > > > around a problem.
> > > >
> > > > What I mean is that get_mbus_config() allows you to retrieve those
> > > > information -without- having to read the remote endpoint, which has
> > > > always been a blocker for this kind of operations (and I agree it
> > > > shouldn't be done).
> >
> > I like get_mbus_config as far as it goes.
> > When it last came up I mentioned that it doesn't convey things like
> > the continuous clock setting, so you have to have the DT endpoints
> > configured identically for that to avoid issues, and if you start to
> > involve CSI2 mux chips then both sources have to have the same
> > requirements.
>
> The merged version of get_mbus_config supports describing
> non-continuous and continuous clock, as we're currently re-using the
> same V4L2_MBUS_ flags the legacy g|s_mbus_config operations used.
OK, I missed that then - sounds good.
My potentially faulty memory said that Hans objected to the older
version passing the continuous clocks flags through as it was all
supposed to be part of DT, but if we're allowed to pass it through
g|s_mbus_config now then all is good.
> > Personally I'd love to see get_mbus_config being the correct way of
> > configuring all receiver parameters as you'd then be guaranteed to
> > have source and receiver matching. Drop back to the receiver's DT
> > endpoint when get_mbus_config isn't supported.
> >
>
> As said, I tend to agree, even if it's a bit freagile having receivers
> and sources agree to -what- has to come from get_mbus_config and from
> DT. But moving forward that would be my preference indeed.
>
> > > > > > Am I missing something maybe? The non-connected lanes on the bride
> > > > > > will simply stay unused, can't they ?
> > > > > >
> > > > > > > implement support for get_mbus_config just yet, it can be delayed until
> > > > > > > we have a source that requires it.
> > > > > >
> > > > > > I sketched a few patches yesterday basically appling what's done in
> > > > > > 5cef0785362ed and a61b1b5d41735. Is a custom property better in your
> > > > > > opinion ?
> > > > >
> > > > > Again, the custom property only specifies the number of data lanes
> > > > > implemented in the unicam IP core. It's nothing else than that, and does
> > > > > *not* convey any information about how many data lanes the sensor will
> > > > > use.
> > > >
> > > > It shouldn't, and to me that information should come from DT as we do
> > > > for all other platforms.
> > > >
> > > > Let's see, if feels a bit a waste to use a custom property for
> > > > something we can do without, and usage of get_mbus_config() would
> > > > allow to support switching devices at run-time with a different number
> > > > of data-lanes, as use case I know Dave has downstream.
> > >
> > > This is *still* not what brcm,num-lanes is about... get_mbus_config()
> > > will not tell you how many data lanes the unicam IP supports.
> >
> > Use case of a TC358743 HDMI to CSI2 bridge connected to Unicam0 (2 lane).
> > TC358743 supports get_mbus_config and says it wants 4 lanes for 1080p60 RGB888.
> > Unicam calls get_mbus_config and gets told 4.
> > So how do we detect and reject that invalid configuration? It can't
>
> unicam0 will have data-lanes=<1 2> and such it's max number of data
> lanes is 2, and if you get 4 from get_mbus_config() you refuse that.
I'd fully back that!
Currently a DT overlay for the 4 lane TC358743 has to set data-lanes
on both TC358743 and Unicam endpoints, so you've now lost the
information on how many lanes Unicam really supports. Changing the
overlay to only update the TC358743 endpoint and Unicam get the config
from get_mbus_config() make life cleaner in my book.
> I now wonder about lane reordering, we can express it in DT but not
> with V4L2_MBUS_* flags. One more reason to supersed these flags with
> more expressive fields, even if lane re-ordering seems to be a corner
> case really.
To my mind the lane order in either sensor or receiver would come from
data-lanes, and the number of active lanes truncates that array to the
specified length.
Are there really use cases where we need to dynamically reorder? I
know some SoCs do mux lanes between multiple receive channels, but I'd
be surprised if there is really a case for doing that on the fly. What
would arbitrate on the lane assignments to avoid conflicts? (Hastily
puts the can of worms back on the shelf as I don't want to get
sidetracked).
> Anyway, I'm a bit sad of upstreaming this with a custom property, but
> this is a broader discussion and I don't want it to block unicam
> acceptance.
We could work without the property, but it removes potential for
validating the configuration. Seeing as we use DT overlays heavily,
and the platforms do support different numbers of lanes, I can see
some really odd bug reports being raised because people don't fully
understand what they are doing.
Migrating away from the custom property as and when the discussions
are concluded shouldn't be too huge a task so I'm happy to go with the
majority.
Dave
> > rely on the Unicam data-lanes as that is supposed to reflect the
> > number of active lanes, but it's not something that can be otherwise
> > detected and blocked. Start streaming and it'll just sit there dumbly
> > doing nothing.
> >
> > Yes we did discuss that most of the Pi boards have only 2 of the 4
> > data lanes from Unicam1 wired out to the camera connector, so whilst
> > technically not what the binding says, it is possible to then drop
> > num-data-lanes down to 2 (though the IP has 4 lanes) and get
> > validation of the config in that case as well.
> >
> > Dave
> >
> > > > > > > > +
> > > > > > > > > + /* Get the local endpoint and remote device. */
> > > > > > > > > + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> > > > > > > > > + if (!ep_node) {
> > > > > > > > > + unicam_dbg(3, dev, "can't get next endpoint\n");
> > > > > > > > > + return -EINVAL;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> > > > > > > > > +
> > > > > > > > > + sensor_node = of_graph_get_remote_port_parent(ep_node);
> > > > > > > > > + if (!sensor_node) {
> > > > > > > > > + unicam_dbg(3, dev, "can't get remote parent\n");
> > > > > > > > > + goto cleanup_exit;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> > > > > > > > > +
> > > > > > > > > + /* Parse the local endpoint and validate its configuration. */
> > > > > > > > > + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> > > > > > > > > +
> > > > > > > > > + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> > > > > > > > > + ep.bus_type);
> > > > > > > > > +
> > > > > > > > > + dev->bus_type = ep.bus_type;
> > > > > > > > > +
> > > > > > > > > + switch (ep.bus_type) {
> > > > > > > > > + case V4L2_MBUS_CSI2_DPHY:
> > > > > > > > > + switch (ep.bus.mipi_csi2.num_data_lanes) {
> > > > > > > > > + case 1:
> > > > > > > > > + case 2:
> > > > > > > > > + case 4:
> > > > > > > > > + break;
> > > > > > > > > +
> > > > > > > > > + default:
> > > > > > > > > + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> > > > > > > > > + sensor_node,
> > > > > > > > > + ep.bus.mipi_csi2.num_data_lanes);
> > > > > > > > > + goto cleanup_exit;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> > > > > > > > > + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> > > > > > > > > + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> > > > > > > > > + sensor_node);
> > > > > > > > > + goto cleanup_exit;
> > > > > > > > > + }
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> > > > > > > > > + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> > > > > > > > > + ep.bus.mipi_csi2.num_data_lanes,
> > > > > > > > > + dev->max_data_lanes);
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> > > > > > > > > + dev->bus_flags = ep.bus.mipi_csi2.flags;
> > > > > > > > > +
> > > > > > > > > + break;
> > > > > > > > > +
> > > > > > > > > + case V4L2_MBUS_CCP2:
> > > > > > > > > + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> > > > > > > > > + ep.bus.mipi_csi1.data_lane != 1) {
> > > > > > > > > + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> > > > > > > > > + sensor_node);
> > > > > > > > > + goto cleanup_exit;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + dev->max_data_lanes = 1;
> > > > > > > > > + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> > > > > > > > > + break;
> > > > > > > > > +
> > > > > > > > > + default:
> > > > > > > > > + /* Unsupported bus type */
> > > > > > > > > + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> > > > > > > > > + sensor_node, ep.bus_type);
> > > > > > > > > + goto cleanup_exit;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> > > > > > > > > + sensor_node,
> > > > > > > > > + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> > > > > > > > > + dev->max_data_lanes, dev->bus_flags);
> > > > > > > > > +
> > > > > > > > > + /* Initialize and register the async notifier. */
> > > > > > > > > + v4l2_async_notifier_init(&dev->notifier);
> > > > > > > > > + dev->notifier.ops = &unicam_async_ops;
> > > > > > > > > +
> > > > > > > > > + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> > > > > > > > > + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> > > > > > > > > + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> > > > > > > > > + if (ret) {
> > > > > > > > > + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> > > > > > > > > + goto cleanup_exit;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> > > > > > > > > + if (ret) {
> > > > > > > > > + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> > > > > > > > > + ret = -EINVAL;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > +cleanup_exit:
> > > > > > > > > + of_node_put(sensor_node);
> > > > > > > > > + of_node_put(ep_node);
> > > > > > > > > +
> > > > > > > > > + return ret;
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +static int unicam_probe(struct platform_device *pdev)
> > > > > > > > > +{
> > > > > > > > > + struct unicam_device *unicam;
> > > > > > > > > + int ret;
> > > > > > > > > +
> > > > > > > > > + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> > > > > > > > > + if (!unicam)
> > > > > > > > > + return -ENOMEM;
> > > > > > > > > +
> > > > > > > > > + kref_init(&unicam->kref);
> > > > > > > > > + unicam->pdev = pdev;
> > > > > > > > > +
> > > > > > > > > + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> > > > > > > > > + if (IS_ERR(unicam->base)) {
> > > > > > > > > + unicam_err(unicam, "Failed to get main io block\n");
> > > > > > > > > + ret = PTR_ERR(unicam->base);
> > > > > > > > > + goto err_unicam_put;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> > > > > > > > > + if (IS_ERR(unicam->clk_gate_base)) {
> > > > > > > > > + unicam_err(unicam, "Failed to get 2nd io block\n");
> > > > > > > > > + ret = PTR_ERR(unicam->clk_gate_base);
> > > > > > > > > + goto err_unicam_put;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> > > > > > > > > + if (IS_ERR(unicam->clock)) {
> > > > > > > > > + unicam_err(unicam, "Failed to get clock\n");
> > > > > > > > > + ret = PTR_ERR(unicam->clock);
> > > > > > > > > + goto err_unicam_put;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + ret = platform_get_irq(pdev, 0);
> > > > > > > > > + if (ret <= 0) {
> > > > > > > > > + dev_err(&pdev->dev, "No IRQ resource\n");
> > > > > > > > > + ret = -EINVAL;
> > > > > > > > > + goto err_unicam_put;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> > > > > > > > > + "unicam_capture0", unicam);
> > > > > > > > > + if (ret) {
> > > > > > > > > + dev_err(&pdev->dev, "Unable to request interrupt\n");
> > > > > > > > > + ret = -EINVAL;
> > > > > > > > > + goto err_unicam_put;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + unicam->mdev.dev = &pdev->dev;
> > > > > > > > > + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> > > > > > > > > + sizeof(unicam->mdev.model));
> > > > > > > > > + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> > > > > > > > > + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> > > > > > > > > + "platform:%s", dev_name(&pdev->dev));
> > > > > > > > > + unicam->mdev.hw_revision = 0;
> > > > > > > > > +
> > > > > > > > > + media_device_init(&unicam->mdev);
> > > > > > > > > +
> > > > > > > > > + unicam->v4l2_dev.mdev = &unicam->mdev;
> > > > > > > > > +
> > > > > > > > > + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> > > > > > > > > + if (ret) {
> > > > > > > > > + unicam_err(unicam,
> > > > > > > > > + "Unable to register v4l2 device.\n");
> > > > > > > > > + goto err_unicam_put;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + ret = media_device_register(&unicam->mdev);
> > > > > > > > > + if (ret < 0) {
> > > > > > > > > + unicam_err(unicam,
> > > > > > > > > + "Unable to register media-controller device.\n");
> > > > > > > > > + goto err_v4l2_unregister;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + /* Reserve space for the controls */
> > > > > > > > > + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> > > > > > > > > + if (ret < 0)
> > > > > > > > > + goto err_media_unregister;
> > > > > > > > > +
> > > > > > > > > + /* set the driver data in platform device */
> > > > > > > > > + platform_set_drvdata(pdev, unicam);
> > > > > > > > > +
> > > > > > > > > + ret = of_unicam_connect_subdevs(unicam);
> > > > > > > > > + if (ret) {
> > > > > > > > > + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> > > > > > > > > + goto err_media_unregister;
> > > > > > > > > + }
> > > > > > > > > +
> > > > > > > > > + /* Enable the block power domain */
> > > > > > > > > + pm_runtime_enable(&pdev->dev);
> > > > > > > > > +
> > > > > > > > > + return 0;
> > > > > > > > > +
> > > > > > > > > +err_media_unregister:
> > > > > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > > > > +err_v4l2_unregister:
> > > > > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > > > > +err_unicam_put:
> > > > > > > > > + unicam_put(unicam);
> > > > > > > > > +
> > > > > > > > > + return ret;
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +static int unicam_remove(struct platform_device *pdev)
> > > > > > > > > +{
> > > > > > > > > + struct unicam_device *unicam = platform_get_drvdata(pdev);
> > > > > > > > > +
> > > > > > > > > + unicam_dbg(2, unicam, "%s\n", __func__);
> > > > > > > > > +
> > > > > > > > > + v4l2_async_notifier_unregister(&unicam->notifier);
> > > > > > > > > + v4l2_device_unregister(&unicam->v4l2_dev);
> > > > > > > > > + media_device_unregister(&unicam->mdev);
> > > > > > > > > + unregister_nodes(unicam);
> > > > > > > > > +
> > > > > > > > > + pm_runtime_disable(&pdev->dev);
> > > > > > > > > +
> > > > > > > > > + return 0;
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +static const struct of_device_id unicam_of_match[] = {
> > > > > > > > > + { .compatible = "brcm,bcm2835-unicam", },
> > > > > > > > > + { /* sentinel */ },
> > > > > > > > > +};
> > > > > > > > > +MODULE_DEVICE_TABLE(of, unicam_of_match);
> > > > > > > > > +
> > > > > > > > > +static struct platform_driver unicam_driver = {
> > > > > > > > > + .probe = unicam_probe,
> > > > > > > > > + .remove = unicam_remove,
> > > > > > > > > + .driver = {
> > > > > > > > > + .name = UNICAM_MODULE_NAME,
> > > > > > > > > + .of_match_table = of_match_ptr(unicam_of_match),
> > > > > > > > > + },
> > > > > > > > > +};
> > > > > > > > > +
> > > > > > > > > +module_platform_driver(unicam_driver);
> > > > > > > > > +
> > > > > > > > > +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> > > > > > > > > +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> > > > > > > > > +MODULE_LICENSE("GPL");
> > > > > > > > > +MODULE_VERSION(UNICAM_VERSION);
> > > > > > > > > diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > > > > new file mode 100644
> > > > > > > > > index 000000000000..ae059a171d0f
> > > > > > > > > --- /dev/null
> > > > > > > > > +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> > > > > > > > > @@ -0,0 +1,253 @@
> > > > > > > > > +/* SPDX-License-Identifier: GPL-2.0-only */
> > > > > > > > > +
> > > > > > > > > +/*
> > > > > > > > > + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> > > > > > > > > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > > > > > > + */
> > > > > > > > > +
> > > > > > > > > +#ifndef VC4_REGS_UNICAM_H
> > > > > > > > > +#define VC4_REGS_UNICAM_H
> > > > > > > > > +
> > > > > > > > > +/*
> > > > > > > > > + * The following values are taken from files found within the code drop
> > > > > > > > > + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> > > > > > > > > + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> > > > > > > > > + * They have been modified to be only the register offset.
> > > > > > > > > + */
> > > > > > > > > +#define UNICAM_CTRL 0x000
> > > > > > > > > +#define UNICAM_STA 0x004
> > > > > > > > > +#define UNICAM_ANA 0x008
> > > > > > > > > +#define UNICAM_PRI 0x00c
> > > > > > > > > +#define UNICAM_CLK 0x010
> > > > > > > > > +#define UNICAM_CLT 0x014
> > > > > > > > > +#define UNICAM_DAT0 0x018
> > > > > > > > > +#define UNICAM_DAT1 0x01c
> > > > > > > > > +#define UNICAM_DAT2 0x020
> > > > > > > > > +#define UNICAM_DAT3 0x024
> > > > > > > > > +#define UNICAM_DLT 0x028
> > > > > > > > > +#define UNICAM_CMP0 0x02c
> > > > > > > > > +#define UNICAM_CMP1 0x030
> > > > > > > > > +#define UNICAM_CAP0 0x034
> > > > > > > > > +#define UNICAM_CAP1 0x038
> > > > > > > > > +#define UNICAM_ICTL 0x100
> > > > > > > > > +#define UNICAM_ISTA 0x104
> > > > > > > > > +#define UNICAM_IDI0 0x108
> > > > > > > > > +#define UNICAM_IPIPE 0x10c
> > > > > > > > > +#define UNICAM_IBSA0 0x110
> > > > > > > > > +#define UNICAM_IBEA0 0x114
> > > > > > > > > +#define UNICAM_IBLS 0x118
> > > > > > > > > +#define UNICAM_IBWP 0x11c
> > > > > > > > > +#define UNICAM_IHWIN 0x120
> > > > > > > > > +#define UNICAM_IHSTA 0x124
> > > > > > > > > +#define UNICAM_IVWIN 0x128
> > > > > > > > > +#define UNICAM_IVSTA 0x12c
> > > > > > > > > +#define UNICAM_ICC 0x130
> > > > > > > > > +#define UNICAM_ICS 0x134
> > > > > > > > > +#define UNICAM_IDC 0x138
> > > > > > > > > +#define UNICAM_IDPO 0x13c
> > > > > > > > > +#define UNICAM_IDCA 0x140
> > > > > > > > > +#define UNICAM_IDCD 0x144
> > > > > > > > > +#define UNICAM_IDS 0x148
> > > > > > > > > +#define UNICAM_DCS 0x200
> > > > > > > > > +#define UNICAM_DBSA0 0x204
> > > > > > > > > +#define UNICAM_DBEA0 0x208
> > > > > > > > > +#define UNICAM_DBWP 0x20c
> > > > > > > > > +#define UNICAM_DBCTL 0x300
> > > > > > > > > +#define UNICAM_IBSA1 0x304
> > > > > > > > > +#define UNICAM_IBEA1 0x308
> > > > > > > > > +#define UNICAM_IDI1 0x30c
> > > > > > > > > +#define UNICAM_DBSA1 0x310
> > > > > > > > > +#define UNICAM_DBEA1 0x314
> > > > > > > > > +#define UNICAM_MISC 0x400
> > > > > > > > > +
> > > > > > > > > +/*
> > > > > > > > > + * The following bitmasks are from the kernel released by Broadcom
> > > > > > > > > + * for Android - https://android.googlesource.com/kernel/bcm/
> > > > > > > > > + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> > > > > > > > > + * Unicam block as BCM2835, as defined in eg
> > > > > > > > > + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> > > > > > > > > + * Values reworked to use the kernel BIT and GENMASK macros.
> > > > > > > > > + *
> > > > > > > > > + * Some of the bit mnenomics have been amended to match the datasheet.
> > > > > > > > > + */
> > > > > > > > > +/* UNICAM_CTRL Register */
> > > > > > > > > +#define UNICAM_CPE BIT(0)
> > > > > > > > > +#define UNICAM_MEM BIT(1)
> > > > > > > > > +#define UNICAM_CPR BIT(2)
> > > > > > > > > +#define UNICAM_CPM_MASK GENMASK(3, 3)
> > > > > > > > > +#define UNICAM_CPM_CSI2 0
> > > > > > > > > +#define UNICAM_CPM_CCP2 1
> > > > > > > > > +#define UNICAM_SOE BIT(4)
> > > > > > > > > +#define UNICAM_DCM_MASK GENMASK(5, 5)
> > > > > > > > > +#define UNICAM_DCM_STROBE 0
> > > > > > > > > +#define UNICAM_DCM_DATA 1
> > > > > > > > > +#define UNICAM_SLS BIT(6)
> > > > > > > > > +#define UNICAM_PFT_MASK GENMASK(11, 8)
> > > > > > > > > +#define UNICAM_OET_MASK GENMASK(20, 12)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_STA Register */
> > > > > > > > > +#define UNICAM_SYN BIT(0)
> > > > > > > > > +#define UNICAM_CS BIT(1)
> > > > > > > > > +#define UNICAM_SBE BIT(2)
> > > > > > > > > +#define UNICAM_PBE BIT(3)
> > > > > > > > > +#define UNICAM_HOE BIT(4)
> > > > > > > > > +#define UNICAM_PLE BIT(5)
> > > > > > > > > +#define UNICAM_SSC BIT(6)
> > > > > > > > > +#define UNICAM_CRCE BIT(7)
> > > > > > > > > +#define UNICAM_OES BIT(8)
> > > > > > > > > +#define UNICAM_IFO BIT(9)
> > > > > > > > > +#define UNICAM_OFO BIT(10)
> > > > > > > > > +#define UNICAM_BFO BIT(11)
> > > > > > > > > +#define UNICAM_DL BIT(12)
> > > > > > > > > +#define UNICAM_PS BIT(13)
> > > > > > > > > +#define UNICAM_IS BIT(14)
> > > > > > > > > +#define UNICAM_PI0 BIT(15)
> > > > > > > > > +#define UNICAM_PI1 BIT(16)
> > > > > > > > > +#define UNICAM_FSI_S BIT(17)
> > > > > > > > > +#define UNICAM_FEI_S BIT(18)
> > > > > > > > > +#define UNICAM_LCI_S BIT(19)
> > > > > > > > > +#define UNICAM_BUF0_RDY BIT(20)
> > > > > > > > > +#define UNICAM_BUF0_NO BIT(21)
> > > > > > > > > +#define UNICAM_BUF1_RDY BIT(22)
> > > > > > > > > +#define UNICAM_BUF1_NO BIT(23)
> > > > > > > > > +#define UNICAM_DI BIT(24)
> > > > > > > > > +
> > > > > > > > > +#define UNICAM_STA_MASK_ALL \
> > > > > > > > > + (UNICAM_DL + \
> > > > > > > > > + UNICAM_SBE + \
> > > > > > > > > + UNICAM_PBE + \
> > > > > > > > > + UNICAM_HOE + \
> > > > > > > > > + UNICAM_PLE + \
> > > > > > > > > + UNICAM_SSC + \
> > > > > > > > > + UNICAM_CRCE + \
> > > > > > > > > + UNICAM_IFO + \
> > > > > > > > > + UNICAM_OFO + \
> > > > > > > > > + UNICAM_PS + \
> > > > > > > > > + UNICAM_PI0 + \
> > > > > > > > > + UNICAM_PI1)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_ANA Register */
> > > > > > > > > +#define UNICAM_APD BIT(0)
> > > > > > > > > +#define UNICAM_BPD BIT(1)
> > > > > > > > > +#define UNICAM_AR BIT(2)
> > > > > > > > > +#define UNICAM_DDL BIT(3)
> > > > > > > > > +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> > > > > > > > > +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_PRI Register */
> > > > > > > > > +#define UNICAM_PE BIT(0)
> > > > > > > > > +#define UNICAM_PT_MASK GENMASK(2, 1)
> > > > > > > > > +#define UNICAM_NP_MASK GENMASK(7, 4)
> > > > > > > > > +#define UNICAM_PP_MASK GENMASK(11, 8)
> > > > > > > > > +#define UNICAM_BS_MASK GENMASK(15, 12)
> > > > > > > > > +#define UNICAM_BL_MASK GENMASK(17, 16)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_CLK Register */
> > > > > > > > > +#define UNICAM_CLE BIT(0)
> > > > > > > > > +#define UNICAM_CLPD BIT(1)
> > > > > > > > > +#define UNICAM_CLLPE BIT(2)
> > > > > > > > > +#define UNICAM_CLHSE BIT(3)
> > > > > > > > > +#define UNICAM_CLTRE BIT(4)
> > > > > > > > > +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> > > > > > > > > +#define UNICAM_CLSTE BIT(29)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_CLT Register */
> > > > > > > > > +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> > > > > > > > > +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_DATn Registers */
> > > > > > > > > +#define UNICAM_DLE BIT(0)
> > > > > > > > > +#define UNICAM_DLPD BIT(1)
> > > > > > > > > +#define UNICAM_DLLPE BIT(2)
> > > > > > > > > +#define UNICAM_DLHSE BIT(3)
> > > > > > > > > +#define UNICAM_DLTRE BIT(4)
> > > > > > > > > +#define UNICAM_DLSM BIT(5)
> > > > > > > > > +#define UNICAM_DLFO BIT(28)
> > > > > > > > > +#define UNICAM_DLSTE BIT(29)
> > > > > > > > > +
> > > > > > > > > +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_DLT Register */
> > > > > > > > > +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> > > > > > > > > +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> > > > > > > > > +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_ICTL Register */
> > > > > > > > > +#define UNICAM_FSIE BIT(0)
> > > > > > > > > +#define UNICAM_FEIE BIT(1)
> > > > > > > > > +#define UNICAM_IBOB BIT(2)
> > > > > > > > > +#define UNICAM_FCM BIT(3)
> > > > > > > > > +#define UNICAM_TFC BIT(4)
> > > > > > > > > +#define UNICAM_LIP_MASK GENMASK(6, 5)
> > > > > > > > > +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_IDI0/1 Register */
> > > > > > > > > +#define UNICAM_ID0_MASK GENMASK(7, 0)
> > > > > > > > > +#define UNICAM_ID1_MASK GENMASK(15, 8)
> > > > > > > > > +#define UNICAM_ID2_MASK GENMASK(23, 16)
> > > > > > > > > +#define UNICAM_ID3_MASK GENMASK(31, 24)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_ISTA Register */
> > > > > > > > > +#define UNICAM_FSI BIT(0)
> > > > > > > > > +#define UNICAM_FEI BIT(1)
> > > > > > > > > +#define UNICAM_LCI BIT(2)
> > > > > > > > > +
> > > > > > > > > +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_IPIPE Register */
> > > > > > > > > +#define UNICAM_PUM_MASK GENMASK(2, 0)
> > > > > > > > > + /* Unpacking modes */
> > > > > > > > > + #define UNICAM_PUM_NONE 0
> > > > > > > > > + #define UNICAM_PUM_UNPACK6 1
> > > > > > > > > + #define UNICAM_PUM_UNPACK7 2
> > > > > > > > > + #define UNICAM_PUM_UNPACK8 3
> > > > > > > > > + #define UNICAM_PUM_UNPACK10 4
> > > > > > > > > + #define UNICAM_PUM_UNPACK12 5
> > > > > > > > > + #define UNICAM_PUM_UNPACK14 6
> > > > > > > > > + #define UNICAM_PUM_UNPACK16 7
> > > > > > > > > +#define UNICAM_DDM_MASK GENMASK(6, 3)
> > > > > > > > > +#define UNICAM_PPM_MASK GENMASK(9, 7)
> > > > > > > > > + /* Packing modes */
> > > > > > > > > + #define UNICAM_PPM_NONE 0
> > > > > > > > > + #define UNICAM_PPM_PACK8 1
> > > > > > > > > + #define UNICAM_PPM_PACK10 2
> > > > > > > > > + #define UNICAM_PPM_PACK12 3
> > > > > > > > > + #define UNICAM_PPM_PACK14 4
> > > > > > > > > + #define UNICAM_PPM_PACK16 5
> > > > > > > > > +#define UNICAM_DEM_MASK GENMASK(11, 10)
> > > > > > > > > +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> > > > > > > > > +#define UNICAM_ICM_MASK GENMASK(16, 15)
> > > > > > > > > +#define UNICAM_IDM_MASK GENMASK(17, 17)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_ICC Register */
> > > > > > > > > +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> > > > > > > > > +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> > > > > > > > > +#define UNICAM_ICST_MASK GENMASK(12, 10)
> > > > > > > > > +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> > > > > > > > > +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_DCS Register */
> > > > > > > > > +#define UNICAM_DIE BIT(0)
> > > > > > > > > +#define UNICAM_DIM BIT(1)
> > > > > > > > > +#define UNICAM_DBOB BIT(3)
> > > > > > > > > +#define UNICAM_FDE BIT(4)
> > > > > > > > > +#define UNICAM_LDP BIT(5)
> > > > > > > > > +#define UNICAM_EDL_MASK GENMASK(15, 8)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_DBCTL Register */
> > > > > > > > > +#define UNICAM_DBEN BIT(0)
> > > > > > > > > +#define UNICAM_BUF0_IE BIT(1)
> > > > > > > > > +#define UNICAM_BUF1_IE BIT(2)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_CMP[0,1] register */
> > > > > > > > > +#define UNICAM_PCE BIT(31)
> > > > > > > > > +#define UNICAM_GI BIT(9)
> > > > > > > > > +#define UNICAM_CPH BIT(8)
> > > > > > > > > +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> > > > > > > > > +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> > > > > > > > > +
> > > > > > > > > +/* UNICAM_MISC register */
> > > > > > > > > +#define UNICAM_FL0 BIT(6)
> > > > > > > > > +#define UNICAM_FL1 BIT(9)
> > > > > > > > > +
> > > > > > > > > +#endif
> > >
> > > --
> > > Regards,
> > >
> > > Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-09-01 16:37 ` Dave Stevenson
@ 2020-09-01 17:11 ` Laurent Pinchart
0 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-09-01 17:11 UTC (permalink / raw)
To: Dave Stevenson
Cc: Jacopo Mondi, Sakari Ailus, Linux Media Mailing List,
Kieran Bingham, Niklas Söderlund, Naushir Patuck
Hi Dave and Jacopo,
On Tue, Sep 01, 2020 at 05:37:28PM +0100, Dave Stevenson wrote:
> On Tue, 1 Sep 2020 at 11:19, Jacopo Mondi wrote:
> > On Tue, Sep 01, 2020 at 09:41:20AM +0100, Dave Stevenson wrote:
> >> On Mon, 31 Aug 2020 at 15:57, Laurent Pinchart wrote:
> >>> On Mon, Aug 31, 2020 at 04:46:37PM +0200, Jacopo Mondi wrote:
> >>>> On Mon, Aug 31, 2020 at 05:17:04PM +0300, Laurent Pinchart wrote:
> >>>>> On Mon, Aug 31, 2020 at 09:38:58AM +0200, Jacopo Mondi wrote:
> >>>>>> On Sat, Aug 29, 2020 at 09:32:37PM +0300, Laurent Pinchart wrote:
> >>>>>>> On Sat, Aug 29, 2020 at 01:20:28PM +0200, Jacopo Mondi wrote:
> >>>>>>>> On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> >>>>>>>>> From: Naushir Patuck <naush@raspberrypi.com>
> >>>>>>>>>
> >>>>>>>>> Add a driver for the Unicam camera receiver block on BCM283x processors.
> >>>>>>>>> Compared to the bcm2835-camera driver present in staging, this driver
> >>>>>>>>> handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> >>>>>>>>> the VC4 firmware running on the VPU.
> >>>>>>>>>
> >>>>>>>>> The commit is made up of a series of changes cherry-picked from the
> >>>>>>>>> rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> >>>>>>>>> additional enhancements, forward-ported to the mainline kernel.
> >>>>>>>>>
> >>>>>>>>> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> >>>>>>>>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> >>>>>>>>> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> >>>>>>>>> ---
> >>>>>>>>> Changes since v1:
> >>>>>>>>>
> >>>>>>>>> - Re-fetch mbus code from subdev on a g_fmt call
> >>>>>>>>> - Group all ioctl disabling together
> >>>>>>>>> - Fix reference counting in unicam_open
> >>>>>>>>> - Add support for VIDIOC_[S|G]_SELECTION
> >>>>>>>>> ---
> >>>>>>>>
> >>>>>>>> [snip]
> >>>>>>>>
> >>>>>>>>> +
> >>>>>>>>> +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> >>>>>>>>> +{
> >>>>>>>>> + struct platform_device *pdev = dev->pdev;
> >>>>>>>>> + struct v4l2_fwnode_endpoint ep = { 0 };
> >>>>>>>>> + struct device_node *ep_node;
> >>>>>>>>> + struct device_node *sensor_node;
> >>>>>>>>> + unsigned int lane;
> >>>>>>>>> + int ret = -EINVAL;
> >>>>>>>>> +
> >>>>>>>>> + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> >>>>>>>>> + &dev->max_data_lanes) < 0) {
> >>>>>>>>> + unicam_err(dev, "number of data lanes not set\n");
> >>>>>>>>> + return -EINVAL;
> >>>>>>>>> + }
> >>>>>>>>
> >>>>>>>> mmm, this seems like a good use case for the newly introduced
> >>>>>>>> get_mbus_config pad operation, isn't it ?
> >>>>>>>>
> >>>>>>>> We could drop the custom "brcm,num-data-lanes" property, use the
> >>>>>>>> standard "num-lanes" to describe the number of wired data lanes, and
> >>>>>>>> get the number of actually used ones at s_stream(1) time with the new pad
> >>>>>>>> operation
> >>>>>>>>
> >>>>>>>> What do you think ? This would require implemeting the operation in
> >>>>>>>> each subdev the UNICAM interface has to work with, but this doesn't
> >>>>>>>> sound too much of a work.
> >>>>>>>>
> >>>>>>>> For reference this is how it happens on rcar+adv748x:
> >>>>>>>> https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-11-jacopo+renesas@jmondi.org/
> >>>>>>>> https://patchwork.linuxtv.org/project/linux-media/patch/20200717145324.292820-10-jacopo+renesas@jmondi.org/
> >>>>>>>
> >>>>>>> Don't brcm,num-data-lanes and get_mbus_config serve different purposes ?
> >>>>>>> The former tells how many data lanes the unicam IP has (the same IP,
> >>>>>>> with the same compatible string, is instantiated in different SoCs with
> >>>>>>> different number of lanes), while the latter queries at runtime how many
> >>>>>>> data lanes the remote subdev will use for its current configuration. The
> >>>>>>
> >>>>>> Can't we get the number of available data lanes just by parsing the
> >>>>>> endpoint ?
> >>>>>
> >>>>> We can get the number of data lanes routed on the board, which may not
> >>>>> match the number of data lanes available in the IP. I'll let Dave
> >>>>> comment on this, I recall we had a lengthy discussion on this topic in
> >>>>> the past, but I don't remember the details :-S
> >>
> >> Thread "[PATCH 10/10] media: bcm2835-unicam: Simplify OF parsing"
> >> posted to the private libcamera list Sun, 19 Apr. I can forward on if
> >> others have deleted it.
> >
> > Thanks. I've now read the thread again.
> >
> > I think my disagreement is with the fact that data-lanes shall
> > represent the number of data lanes connected between the sensor and the
> > receiver.
> >
> > data-lanes, to me, represents the number (and index) of the lanes
> > instantiated on the silicon, or even exposed on the connector. In
> > plain English: "the number of available data lanes".
>
> That would be my wish too, but it's not the current usage.
I agree with Dave here. The data-lanes property in the endpoint
describes the board. That is very unlikely to change.
> > The sensor will describe it's own lanes (which might be less than the
> > ones available in the receiver) and at streaming time the receiver
> > gets how many of them shall be used by using get_mbus_config(). This
> > would basically move the notion of "how many lanes are in use" to
> > run-time, leaving the HW description of "what's available in the HW"
> > to DT. It seems a much neater solution to me, also considering the
> > current implementation does not scale if we throw muxes and pluggable
> > devices in the mix.
>
> Yes, if get_mbus_config() was mandatory for all sensor/source drivers
> to implement then that would work, but currently I think there are
> only about 4 drivers that implement it.
We could extend that, but I don't think there's a need to.
.get_mbus_config() is primarly meant to support CSI-2 sources that use
different number of data lanes depending on their configuration. For
sources that will always use the number of data lanes routed on the PCB,
the information available in DT is enough.
> It'd be nice if there was a way for the framework to take care of it
> for the simple drivers that purely take the config from DT, but I'm
> not sure that's feasible.
>
> > This isn't a discussion on this patch only, and I know Laurent and
> > Sakari already had gone through that multiple times. I see if I have
> > energies to fight that battle, but for this series let's live with the
> > custom property.
> >
> >>>> I think the max number of number of lanes in the IP should come in DT
> >>>> by restricting the number of accepted data-lanes value depending on
> >>>> the compatible string.
> >>>
> >>> Unless I'm mistaken we have a single compatibility string.
> >>
> >> Currently we have a single compatible string.
> >> So you want a "brcm,bcm2835-unicam-2-lane" and "brcm,bcm2835-unicam-4-lane"?
> >
> > No please :) I assumed you had different ones.
>
> I'm glad you said that :-)
> Other than the number of lanes the IP in all the variations of Pi is
> identical, so adding multiple compatibles seemed excessive. It's the
> same for a fair number of the IP blocks in the SoC such as I2C and
> SPI.
I fully agree. The number of lanes supported by an instance of the
unicam IP core is a property of the core, and should be reported in DT
if the driver needs to access that information.
> >> You can't guarantee that unicam0 is always 2 lane and unicam1 is
> >> always 4 lane forever more - I know there are Broadcom chips with a
> >> unicam2 for a start.
> >> I defer to the DT gatekeepers, but that feels ugly.
It would be ugly, let's not go down that route.
> >>>>>> I'm aware of the implication that the 'data-lanes' property
> >>>>>> would differ in the bridge and sensor endpoints, but now that we have
> >>>>>> a mechanism to get that information wihtout parsing the remote
> >>>>>> endpoint it doesn't seem wrong to me.
> >>>>>
> >>>>> That's not the issue at hand. The data-lanes property need to match on
> >>>>> both sides of a link, as it tells how many data lanes are routed on the
> >>>>> board. Of course that becomes problematic when we start using overlays
> >>>>> for pluggable boards, as the two sides can be designed with a different
> >>>>> number of routed lanes.
> >>>>
> >>>> It actually is. As I read the current implementation, UNICAM has 4
> >>>> lanes available, 2 are enabled in the endpoint, and the remote uses 2.
> >>>
> >>> That's because the board has 2 data lanes routed.
> >>>
> >>>> If we set the number of lanes in the endpoint to 4 (the actually
> >>>> available ones) and use only the two required by querying the remote
> >>>> end at s_stream time, we don't need any custom sourcery
> >>>
> >>> That would be incorrect, as there are two data lanes routed, so you
> >>> shouldn't set the data-lanes property to 4 lanes in the endpoint.
> >>>
> >>>>>> And that's not different from what it's done in R-Car (half-true: we
> >>>>>> have the same 'data-lanes' value between rcar-csi2 and adv7482x, and
> >>>>>> as you said the number of active data lanes can be downscaled to
> >>>>>> transmit low resolution streams).
> >>>>>>
> >>>>>> I think Dave already had this conversation in the past, I might dig
> >>>>>> the archive and search for a reference.
> >>
> >> Referenced above.
> >>
> >>>>>>> DT property serves as a sanity check, to ensure to no more than the
> >>>>>>> number of supported data lanes is used, while the subdev operation
> >>>>>>> should be used in the unicam driver to support sources that will
> >>>>>>> transmit over a different number of data lanes depending on the
> >>>>>>> configuration (mostly depending on the resolution). We don't have to
> >>>>>>
> >>>>>> I think reading the remote's mbus configuration might also help in
> >>>>>> support connecting pluggable devices with a different number of wired
> >>>>>> data lanes than the bridge's available ones.
> >>>>>
> >>>>> No, that we shouldn't do. DT properties need to be interpreted within
> >>>>> the context of a compatible string, reading properties of a remote node
> >>>>> is a no-go in most cases. It's really really something that must not be
> >>>>> done without careful consideration, and is never a simple way to work
> >>>>> around a problem.
> >>>>
> >>>> What I mean is that get_mbus_config() allows you to retrieve those
> >>>> information -without- having to read the remote endpoint, which has
> >>>> always been a blocker for this kind of operations (and I agree it
> >>>> shouldn't be done).
> >>
> >> I like get_mbus_config as far as it goes.
> >> When it last came up I mentioned that it doesn't convey things like
> >> the continuous clock setting, so you have to have the DT endpoints
> >> configured identically for that to avoid issues, and if you start to
> >> involve CSI2 mux chips then both sources have to have the same
> >> requirements.
> >
> > The merged version of get_mbus_config supports describing
> > non-continuous and continuous clock, as we're currently re-using the
> > same V4L2_MBUS_ flags the legacy g|s_mbus_config operations used.
>
> OK, I missed that then - sounds good.
> My potentially faulty memory said that Hans objected to the older
> version passing the continuous clocks flags through as it was all
> supposed to be part of DT, but if we're allowed to pass it through
> g|s_mbus_config now then all is good.
The clock mode is something that shouldn't have been in DT in the first
place I believe (with the exception of describing what modes a
particular instance of a device supports, the same way we do for the
number of data lanes, but that would be at the device level, not the
endpoint level), it should have been negotiated at runtime between the
source and sink. With .get_mbus_config() we can possibly start moving in
that direction.
> >> Personally I'd love to see get_mbus_config being the correct way of
> >> configuring all receiver parameters as you'd then be guaranteed to
> >> have source and receiver matching. Drop back to the receiver's DT
> >> endpoint when get_mbus_config isn't supported.
> >
> > As said, I tend to agree, even if it's a bit freagile having receivers
> > and sources agree to -what- has to come from get_mbus_config and from
> > DT. But moving forward that would be my preference indeed.
We can't fully rely on .get_mbus_config() though. At the least clock and
data lanes mapping needs to be specified in DT on the receiver side (for
receivers that support configurable mappings), as it's a description of
the board for the receiver side, so the transmitter should need to care
(the transmitter however needs to handle the same on its side, if it
supports configurable mappings).
> >>>>>> Am I missing something maybe? The non-connected lanes on the bride
> >>>>>> will simply stay unused, can't they ?
> >>>>>>
> >>>>>>> implement support for get_mbus_config just yet, it can be delayed until
> >>>>>>> we have a source that requires it.
> >>>>>>
> >>>>>> I sketched a few patches yesterday basically appling what's done in
> >>>>>> 5cef0785362ed and a61b1b5d41735. Is a custom property better in your
> >>>>>> opinion ?
> >>>>>
> >>>>> Again, the custom property only specifies the number of data lanes
> >>>>> implemented in the unicam IP core. It's nothing else than that, and does
> >>>>> *not* convey any information about how many data lanes the sensor will
> >>>>> use.
> >>>>
> >>>> It shouldn't, and to me that information should come from DT as we do
> >>>> for all other platforms.
> >>>>
> >>>> Let's see, if feels a bit a waste to use a custom property for
> >>>> something we can do without, and usage of get_mbus_config() would
> >>>> allow to support switching devices at run-time with a different number
> >>>> of data-lanes, as use case I know Dave has downstream.
> >>>
> >>> This is *still* not what brcm,num-lanes is about... get_mbus_config()
> >>> will not tell you how many data lanes the unicam IP supports.
> >>
> >> Use case of a TC358743 HDMI to CSI2 bridge connected to Unicam0 (2 lane).
> >> TC358743 supports get_mbus_config and says it wants 4 lanes for 1080p60 RGB888.
> >> Unicam calls get_mbus_config and gets told 4.
> >> So how do we detect and reject that invalid configuration? It can't
> >
> > unicam0 will have data-lanes=<1 2> and such it's max number of data
> > lanes is 2, and if you get 4 from get_mbus_config() you refuse that.
>
> I'd fully back that!
>
> Currently a DT overlay for the 4 lane TC358743 has to set data-lanes
> on both TC358743 and Unicam endpoints, so you've now lost the
> information on how many lanes Unicam really supports. Changing the
> overlay to only update the TC358743 endpoint and Unicam get the config
> from get_mbus_config() make life cleaner in my book.
The whole issue here is I believe a side effect of using overlays.
Without overlays, you would simply set data-lanes = <1 2> on both sides,
as that's what is routed on the PCB. With overlays, if you want a
generic overlay for the TC358743 module, we start having issues.
Even with .get_mbus_config() we don't have a full solution for this
problem. The TC358743 needs to know that only two lanes are routed to
the receiver in order to configure itself correctly (and reject invalid
configurations). If we don't provide that through DT (as data-lanes is
set to <1 2 3 4> in the TC358743 overlay), then the information would
need to be provided to the TC358743 driver at runtime.
I would like to explore a technical solution to have data-lanes set to
<1 2> on both sides as the result of merging the overlay with the base
DT file. That would solve the issue, and keep drivers simple(r). I'm not
sure how it could be achieved though.
> > I now wonder about lane reordering, we can express it in DT but not
> > with V4L2_MBUS_* flags. One more reason to supersed these flags with
> > more expressive fields, even if lane re-ordering seems to be a corner
> > case really.
>
> To my mind the lane order in either sensor or receiver would come from
> data-lanes, and the number of active lanes truncates that array to the
> specified length.
Lane ordering should be a local configuration on both sides (the CSI-2
receiver doesn't need to know how the lanes are reordered on the
transmitter side, and vice-versa, as long as they share an understanding
of how lanes are numbered on the PCB).
> Are there really use cases where we need to dynamically reorder? I
> know some SoCs do mux lanes between multiple receive channels, but I'd
> be surprised if there is really a case for doing that on the fly. What
> would arbitrate on the lane assignments to avoid conflicts? (Hastily
> puts the can of worms back on the shelf as I don't want to get
> sidetracked).
I think there's a theoretical use case, and I'd like to keep it
theoretical :-)
> > Anyway, I'm a bit sad of upstreaming this with a custom property, but
> > this is a broader discussion and I don't want it to block unicam
> > acceptance.
I'm sorry, but I see nothing wrong with this property. It describes one
of the synthesis parameters for the IP core, and DT is the perfect place
to specify this. Look at the Xilinx bindings for instance, they specify
lots of synthesis parameters in DT. It's the perfect example of a
hardware description for an IP core.
> We could work without the property, but it removes potential for
> validating the configuration.
This point I'm certainly willing to discuss. We could omit the
brcm,num-data-lanes property if we considered that there would be no use
for this information in drivers. At the moment the value is used for
validation, and I think it makes sense as a use case, but I'm open to
reconsidering this if the consensus is that this feature isn't very
useful.
> Seeing as we use DT overlays heavily,
> and the platforms do support different numbers of lanes, I can see
> some really odd bug reports being raised because people don't fully
> understand what they are doing.
> Migrating away from the custom property as and when the discussions
> are concluded shouldn't be too huge a task so I'm happy to go with the
> majority.
I agree with you, I think the validation is useful at the moment. I
however don't think brcm,num-data-lanes should be used to lower the
number of lanes at runtime, as the CSI-2 source wouldn't know about
that. Let's see if we can find a solution to set the endpoint data-lanes
property to reference two lanes on both sides while keeping overlays
generic.
> >> rely on the Unicam data-lanes as that is supposed to reflect the
> >> number of active lanes, but it's not something that can be otherwise
> >> detected and blocked. Start streaming and it'll just sit there dumbly
> >> doing nothing.
> >>
> >> Yes we did discuss that most of the Pi boards have only 2 of the 4
> >> data lanes from Unicam1 wired out to the camera connector, so whilst
> >> technically not what the binding says, it is possible to then drop
> >> num-data-lanes down to 2 (though the IP has 4 lanes) and get
> >> validation of the config in that case as well.
> >>
> >>>>>>>> +
> >>>>>>>>> + /* Get the local endpoint and remote device. */
> >>>>>>>>> + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> >>>>>>>>> + if (!ep_node) {
> >>>>>>>>> + unicam_dbg(3, dev, "can't get next endpoint\n");
> >>>>>>>>> + return -EINVAL;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
> >>>>>>>>> +
> >>>>>>>>> + sensor_node = of_graph_get_remote_port_parent(ep_node);
> >>>>>>>>> + if (!sensor_node) {
> >>>>>>>>> + unicam_dbg(3, dev, "can't get remote parent\n");
> >>>>>>>>> + goto cleanup_exit;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> >>>>>>>>> +
> >>>>>>>>> + /* Parse the local endpoint and validate its configuration. */
> >>>>>>>>> + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> >>>>>>>>> +
> >>>>>>>>> + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> >>>>>>>>> + ep.bus_type);
> >>>>>>>>> +
> >>>>>>>>> + dev->bus_type = ep.bus_type;
> >>>>>>>>> +
> >>>>>>>>> + switch (ep.bus_type) {
> >>>>>>>>> + case V4L2_MBUS_CSI2_DPHY:
> >>>>>>>>> + switch (ep.bus.mipi_csi2.num_data_lanes) {
> >>>>>>>>> + case 1:
> >>>>>>>>> + case 2:
> >>>>>>>>> + case 4:
> >>>>>>>>> + break;
> >>>>>>>>> +
> >>>>>>>>> + default:
> >>>>>>>>> + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> >>>>>>>>> + sensor_node,
> >>>>>>>>> + ep.bus.mipi_csi2.num_data_lanes);
> >>>>>>>>> + goto cleanup_exit;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> >>>>>>>>> + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> >>>>>>>>> + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> >>>>>>>>> + sensor_node);
> >>>>>>>>> + goto cleanup_exit;
> >>>>>>>>> + }
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> >>>>>>>>> + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> >>>>>>>>> + ep.bus.mipi_csi2.num_data_lanes,
> >>>>>>>>> + dev->max_data_lanes);
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> >>>>>>>>> + dev->bus_flags = ep.bus.mipi_csi2.flags;
> >>>>>>>>> +
> >>>>>>>>> + break;
> >>>>>>>>> +
> >>>>>>>>> + case V4L2_MBUS_CCP2:
> >>>>>>>>> + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> >>>>>>>>> + ep.bus.mipi_csi1.data_lane != 1) {
> >>>>>>>>> + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> >>>>>>>>> + sensor_node);
> >>>>>>>>> + goto cleanup_exit;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + dev->max_data_lanes = 1;
> >>>>>>>>> + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> >>>>>>>>> + break;
> >>>>>>>>> +
> >>>>>>>>> + default:
> >>>>>>>>> + /* Unsupported bus type */
> >>>>>>>>> + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> >>>>>>>>> + sensor_node, ep.bus_type);
> >>>>>>>>> + goto cleanup_exit;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> >>>>>>>>> + sensor_node,
> >>>>>>>>> + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> >>>>>>>>> + dev->max_data_lanes, dev->bus_flags);
> >>>>>>>>> +
> >>>>>>>>> + /* Initialize and register the async notifier. */
> >>>>>>>>> + v4l2_async_notifier_init(&dev->notifier);
> >>>>>>>>> + dev->notifier.ops = &unicam_async_ops;
> >>>>>>>>> +
> >>>>>>>>> + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> >>>>>>>>> + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> >>>>>>>>> + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
> >>>>>>>>> + if (ret) {
> >>>>>>>>> + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> >>>>>>>>> + goto cleanup_exit;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> >>>>>>>>> + if (ret) {
> >>>>>>>>> + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> >>>>>>>>> + ret = -EINVAL;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> +cleanup_exit:
> >>>>>>>>> + of_node_put(sensor_node);
> >>>>>>>>> + of_node_put(ep_node);
> >>>>>>>>> +
> >>>>>>>>> + return ret;
> >>>>>>>>> +}
> >>>>>>>>> +
> >>>>>>>>> +static int unicam_probe(struct platform_device *pdev)
> >>>>>>>>> +{
> >>>>>>>>> + struct unicam_device *unicam;
> >>>>>>>>> + int ret;
> >>>>>>>>> +
> >>>>>>>>> + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
> >>>>>>>>> + if (!unicam)
> >>>>>>>>> + return -ENOMEM;
> >>>>>>>>> +
> >>>>>>>>> + kref_init(&unicam->kref);
> >>>>>>>>> + unicam->pdev = pdev;
> >>>>>>>>> +
> >>>>>>>>> + unicam->base = devm_platform_ioremap_resource(pdev, 0);
> >>>>>>>>> + if (IS_ERR(unicam->base)) {
> >>>>>>>>> + unicam_err(unicam, "Failed to get main io block\n");
> >>>>>>>>> + ret = PTR_ERR(unicam->base);
> >>>>>>>>> + goto err_unicam_put;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> >>>>>>>>> + if (IS_ERR(unicam->clk_gate_base)) {
> >>>>>>>>> + unicam_err(unicam, "Failed to get 2nd io block\n");
> >>>>>>>>> + ret = PTR_ERR(unicam->clk_gate_base);
> >>>>>>>>> + goto err_unicam_put;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> >>>>>>>>> + if (IS_ERR(unicam->clock)) {
> >>>>>>>>> + unicam_err(unicam, "Failed to get clock\n");
> >>>>>>>>> + ret = PTR_ERR(unicam->clock);
> >>>>>>>>> + goto err_unicam_put;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + ret = platform_get_irq(pdev, 0);
> >>>>>>>>> + if (ret <= 0) {
> >>>>>>>>> + dev_err(&pdev->dev, "No IRQ resource\n");
> >>>>>>>>> + ret = -EINVAL;
> >>>>>>>>> + goto err_unicam_put;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> >>>>>>>>> + "unicam_capture0", unicam);
> >>>>>>>>> + if (ret) {
> >>>>>>>>> + dev_err(&pdev->dev, "Unable to request interrupt\n");
> >>>>>>>>> + ret = -EINVAL;
> >>>>>>>>> + goto err_unicam_put;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + unicam->mdev.dev = &pdev->dev;
> >>>>>>>>> + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
> >>>>>>>>> + sizeof(unicam->mdev.model));
> >>>>>>>>> + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> >>>>>>>>> + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> >>>>>>>>> + "platform:%s", dev_name(&pdev->dev));
> >>>>>>>>> + unicam->mdev.hw_revision = 0;
> >>>>>>>>> +
> >>>>>>>>> + media_device_init(&unicam->mdev);
> >>>>>>>>> +
> >>>>>>>>> + unicam->v4l2_dev.mdev = &unicam->mdev;
> >>>>>>>>> +
> >>>>>>>>> + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> >>>>>>>>> + if (ret) {
> >>>>>>>>> + unicam_err(unicam,
> >>>>>>>>> + "Unable to register v4l2 device.\n");
> >>>>>>>>> + goto err_unicam_put;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + ret = media_device_register(&unicam->mdev);
> >>>>>>>>> + if (ret < 0) {
> >>>>>>>>> + unicam_err(unicam,
> >>>>>>>>> + "Unable to register media-controller device.\n");
> >>>>>>>>> + goto err_v4l2_unregister;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + /* Reserve space for the controls */
> >>>>>>>>> + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> >>>>>>>>> + if (ret < 0)
> >>>>>>>>> + goto err_media_unregister;
> >>>>>>>>> +
> >>>>>>>>> + /* set the driver data in platform device */
> >>>>>>>>> + platform_set_drvdata(pdev, unicam);
> >>>>>>>>> +
> >>>>>>>>> + ret = of_unicam_connect_subdevs(unicam);
> >>>>>>>>> + if (ret) {
> >>>>>>>>> + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> >>>>>>>>> + goto err_media_unregister;
> >>>>>>>>> + }
> >>>>>>>>> +
> >>>>>>>>> + /* Enable the block power domain */
> >>>>>>>>> + pm_runtime_enable(&pdev->dev);
> >>>>>>>>> +
> >>>>>>>>> + return 0;
> >>>>>>>>> +
> >>>>>>>>> +err_media_unregister:
> >>>>>>>>> + media_device_unregister(&unicam->mdev);
> >>>>>>>>> +err_v4l2_unregister:
> >>>>>>>>> + v4l2_device_unregister(&unicam->v4l2_dev);
> >>>>>>>>> +err_unicam_put:
> >>>>>>>>> + unicam_put(unicam);
> >>>>>>>>> +
> >>>>>>>>> + return ret;
> >>>>>>>>> +}
> >>>>>>>>> +
> >>>>>>>>> +static int unicam_remove(struct platform_device *pdev)
> >>>>>>>>> +{
> >>>>>>>>> + struct unicam_device *unicam = platform_get_drvdata(pdev);
> >>>>>>>>> +
> >>>>>>>>> + unicam_dbg(2, unicam, "%s\n", __func__);
> >>>>>>>>> +
> >>>>>>>>> + v4l2_async_notifier_unregister(&unicam->notifier);
> >>>>>>>>> + v4l2_device_unregister(&unicam->v4l2_dev);
> >>>>>>>>> + media_device_unregister(&unicam->mdev);
> >>>>>>>>> + unregister_nodes(unicam);
> >>>>>>>>> +
> >>>>>>>>> + pm_runtime_disable(&pdev->dev);
> >>>>>>>>> +
> >>>>>>>>> + return 0;
> >>>>>>>>> +}
> >>>>>>>>> +
> >>>>>>>>> +static const struct of_device_id unicam_of_match[] = {
> >>>>>>>>> + { .compatible = "brcm,bcm2835-unicam", },
> >>>>>>>>> + { /* sentinel */ },
> >>>>>>>>> +};
> >>>>>>>>> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> >>>>>>>>> +
> >>>>>>>>> +static struct platform_driver unicam_driver = {
> >>>>>>>>> + .probe = unicam_probe,
> >>>>>>>>> + .remove = unicam_remove,
> >>>>>>>>> + .driver = {
> >>>>>>>>> + .name = UNICAM_MODULE_NAME,
> >>>>>>>>> + .of_match_table = of_match_ptr(unicam_of_match),
> >>>>>>>>> + },
> >>>>>>>>> +};
> >>>>>>>>> +
> >>>>>>>>> +module_platform_driver(unicam_driver);
> >>>>>>>>> +
> >>>>>>>>> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> >>>>>>>>> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> >>>>>>>>> +MODULE_LICENSE("GPL");
> >>>>>>>>> +MODULE_VERSION(UNICAM_VERSION);
> >>>>>>>>> diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> >>>>>>>>> new file mode 100644
> >>>>>>>>> index 000000000000..ae059a171d0f
> >>>>>>>>> --- /dev/null
> >>>>>>>>> +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> >>>>>>>>> @@ -0,0 +1,253 @@
> >>>>>>>>> +/* SPDX-License-Identifier: GPL-2.0-only */
> >>>>>>>>> +
> >>>>>>>>> +/*
> >>>>>>>>> + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> >>>>>>>>> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> >>>>>>>>> + */
> >>>>>>>>> +
> >>>>>>>>> +#ifndef VC4_REGS_UNICAM_H
> >>>>>>>>> +#define VC4_REGS_UNICAM_H
> >>>>>>>>> +
> >>>>>>>>> +/*
> >>>>>>>>> + * The following values are taken from files found within the code drop
> >>>>>>>>> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> >>>>>>>>> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> >>>>>>>>> + * They have been modified to be only the register offset.
> >>>>>>>>> + */
> >>>>>>>>> +#define UNICAM_CTRL 0x000
> >>>>>>>>> +#define UNICAM_STA 0x004
> >>>>>>>>> +#define UNICAM_ANA 0x008
> >>>>>>>>> +#define UNICAM_PRI 0x00c
> >>>>>>>>> +#define UNICAM_CLK 0x010
> >>>>>>>>> +#define UNICAM_CLT 0x014
> >>>>>>>>> +#define UNICAM_DAT0 0x018
> >>>>>>>>> +#define UNICAM_DAT1 0x01c
> >>>>>>>>> +#define UNICAM_DAT2 0x020
> >>>>>>>>> +#define UNICAM_DAT3 0x024
> >>>>>>>>> +#define UNICAM_DLT 0x028
> >>>>>>>>> +#define UNICAM_CMP0 0x02c
> >>>>>>>>> +#define UNICAM_CMP1 0x030
> >>>>>>>>> +#define UNICAM_CAP0 0x034
> >>>>>>>>> +#define UNICAM_CAP1 0x038
> >>>>>>>>> +#define UNICAM_ICTL 0x100
> >>>>>>>>> +#define UNICAM_ISTA 0x104
> >>>>>>>>> +#define UNICAM_IDI0 0x108
> >>>>>>>>> +#define UNICAM_IPIPE 0x10c
> >>>>>>>>> +#define UNICAM_IBSA0 0x110
> >>>>>>>>> +#define UNICAM_IBEA0 0x114
> >>>>>>>>> +#define UNICAM_IBLS 0x118
> >>>>>>>>> +#define UNICAM_IBWP 0x11c
> >>>>>>>>> +#define UNICAM_IHWIN 0x120
> >>>>>>>>> +#define UNICAM_IHSTA 0x124
> >>>>>>>>> +#define UNICAM_IVWIN 0x128
> >>>>>>>>> +#define UNICAM_IVSTA 0x12c
> >>>>>>>>> +#define UNICAM_ICC 0x130
> >>>>>>>>> +#define UNICAM_ICS 0x134
> >>>>>>>>> +#define UNICAM_IDC 0x138
> >>>>>>>>> +#define UNICAM_IDPO 0x13c
> >>>>>>>>> +#define UNICAM_IDCA 0x140
> >>>>>>>>> +#define UNICAM_IDCD 0x144
> >>>>>>>>> +#define UNICAM_IDS 0x148
> >>>>>>>>> +#define UNICAM_DCS 0x200
> >>>>>>>>> +#define UNICAM_DBSA0 0x204
> >>>>>>>>> +#define UNICAM_DBEA0 0x208
> >>>>>>>>> +#define UNICAM_DBWP 0x20c
> >>>>>>>>> +#define UNICAM_DBCTL 0x300
> >>>>>>>>> +#define UNICAM_IBSA1 0x304
> >>>>>>>>> +#define UNICAM_IBEA1 0x308
> >>>>>>>>> +#define UNICAM_IDI1 0x30c
> >>>>>>>>> +#define UNICAM_DBSA1 0x310
> >>>>>>>>> +#define UNICAM_DBEA1 0x314
> >>>>>>>>> +#define UNICAM_MISC 0x400
> >>>>>>>>> +
> >>>>>>>>> +/*
> >>>>>>>>> + * The following bitmasks are from the kernel released by Broadcom
> >>>>>>>>> + * for Android - https://android.googlesource.com/kernel/bcm/
> >>>>>>>>> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> >>>>>>>>> + * Unicam block as BCM2835, as defined in eg
> >>>>>>>>> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> >>>>>>>>> + * Values reworked to use the kernel BIT and GENMASK macros.
> >>>>>>>>> + *
> >>>>>>>>> + * Some of the bit mnenomics have been amended to match the datasheet.
> >>>>>>>>> + */
> >>>>>>>>> +/* UNICAM_CTRL Register */
> >>>>>>>>> +#define UNICAM_CPE BIT(0)
> >>>>>>>>> +#define UNICAM_MEM BIT(1)
> >>>>>>>>> +#define UNICAM_CPR BIT(2)
> >>>>>>>>> +#define UNICAM_CPM_MASK GENMASK(3, 3)
> >>>>>>>>> +#define UNICAM_CPM_CSI2 0
> >>>>>>>>> +#define UNICAM_CPM_CCP2 1
> >>>>>>>>> +#define UNICAM_SOE BIT(4)
> >>>>>>>>> +#define UNICAM_DCM_MASK GENMASK(5, 5)
> >>>>>>>>> +#define UNICAM_DCM_STROBE 0
> >>>>>>>>> +#define UNICAM_DCM_DATA 1
> >>>>>>>>> +#define UNICAM_SLS BIT(6)
> >>>>>>>>> +#define UNICAM_PFT_MASK GENMASK(11, 8)
> >>>>>>>>> +#define UNICAM_OET_MASK GENMASK(20, 12)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_STA Register */
> >>>>>>>>> +#define UNICAM_SYN BIT(0)
> >>>>>>>>> +#define UNICAM_CS BIT(1)
> >>>>>>>>> +#define UNICAM_SBE BIT(2)
> >>>>>>>>> +#define UNICAM_PBE BIT(3)
> >>>>>>>>> +#define UNICAM_HOE BIT(4)
> >>>>>>>>> +#define UNICAM_PLE BIT(5)
> >>>>>>>>> +#define UNICAM_SSC BIT(6)
> >>>>>>>>> +#define UNICAM_CRCE BIT(7)
> >>>>>>>>> +#define UNICAM_OES BIT(8)
> >>>>>>>>> +#define UNICAM_IFO BIT(9)
> >>>>>>>>> +#define UNICAM_OFO BIT(10)
> >>>>>>>>> +#define UNICAM_BFO BIT(11)
> >>>>>>>>> +#define UNICAM_DL BIT(12)
> >>>>>>>>> +#define UNICAM_PS BIT(13)
> >>>>>>>>> +#define UNICAM_IS BIT(14)
> >>>>>>>>> +#define UNICAM_PI0 BIT(15)
> >>>>>>>>> +#define UNICAM_PI1 BIT(16)
> >>>>>>>>> +#define UNICAM_FSI_S BIT(17)
> >>>>>>>>> +#define UNICAM_FEI_S BIT(18)
> >>>>>>>>> +#define UNICAM_LCI_S BIT(19)
> >>>>>>>>> +#define UNICAM_BUF0_RDY BIT(20)
> >>>>>>>>> +#define UNICAM_BUF0_NO BIT(21)
> >>>>>>>>> +#define UNICAM_BUF1_RDY BIT(22)
> >>>>>>>>> +#define UNICAM_BUF1_NO BIT(23)
> >>>>>>>>> +#define UNICAM_DI BIT(24)
> >>>>>>>>> +
> >>>>>>>>> +#define UNICAM_STA_MASK_ALL \
> >>>>>>>>> + (UNICAM_DL + \
> >>>>>>>>> + UNICAM_SBE + \
> >>>>>>>>> + UNICAM_PBE + \
> >>>>>>>>> + UNICAM_HOE + \
> >>>>>>>>> + UNICAM_PLE + \
> >>>>>>>>> + UNICAM_SSC + \
> >>>>>>>>> + UNICAM_CRCE + \
> >>>>>>>>> + UNICAM_IFO + \
> >>>>>>>>> + UNICAM_OFO + \
> >>>>>>>>> + UNICAM_PS + \
> >>>>>>>>> + UNICAM_PI0 + \
> >>>>>>>>> + UNICAM_PI1)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_ANA Register */
> >>>>>>>>> +#define UNICAM_APD BIT(0)
> >>>>>>>>> +#define UNICAM_BPD BIT(1)
> >>>>>>>>> +#define UNICAM_AR BIT(2)
> >>>>>>>>> +#define UNICAM_DDL BIT(3)
> >>>>>>>>> +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> >>>>>>>>> +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_PRI Register */
> >>>>>>>>> +#define UNICAM_PE BIT(0)
> >>>>>>>>> +#define UNICAM_PT_MASK GENMASK(2, 1)
> >>>>>>>>> +#define UNICAM_NP_MASK GENMASK(7, 4)
> >>>>>>>>> +#define UNICAM_PP_MASK GENMASK(11, 8)
> >>>>>>>>> +#define UNICAM_BS_MASK GENMASK(15, 12)
> >>>>>>>>> +#define UNICAM_BL_MASK GENMASK(17, 16)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_CLK Register */
> >>>>>>>>> +#define UNICAM_CLE BIT(0)
> >>>>>>>>> +#define UNICAM_CLPD BIT(1)
> >>>>>>>>> +#define UNICAM_CLLPE BIT(2)
> >>>>>>>>> +#define UNICAM_CLHSE BIT(3)
> >>>>>>>>> +#define UNICAM_CLTRE BIT(4)
> >>>>>>>>> +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> >>>>>>>>> +#define UNICAM_CLSTE BIT(29)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_CLT Register */
> >>>>>>>>> +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> >>>>>>>>> +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_DATn Registers */
> >>>>>>>>> +#define UNICAM_DLE BIT(0)
> >>>>>>>>> +#define UNICAM_DLPD BIT(1)
> >>>>>>>>> +#define UNICAM_DLLPE BIT(2)
> >>>>>>>>> +#define UNICAM_DLHSE BIT(3)
> >>>>>>>>> +#define UNICAM_DLTRE BIT(4)
> >>>>>>>>> +#define UNICAM_DLSM BIT(5)
> >>>>>>>>> +#define UNICAM_DLFO BIT(28)
> >>>>>>>>> +#define UNICAM_DLSTE BIT(29)
> >>>>>>>>> +
> >>>>>>>>> +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_DLT Register */
> >>>>>>>>> +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> >>>>>>>>> +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> >>>>>>>>> +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_ICTL Register */
> >>>>>>>>> +#define UNICAM_FSIE BIT(0)
> >>>>>>>>> +#define UNICAM_FEIE BIT(1)
> >>>>>>>>> +#define UNICAM_IBOB BIT(2)
> >>>>>>>>> +#define UNICAM_FCM BIT(3)
> >>>>>>>>> +#define UNICAM_TFC BIT(4)
> >>>>>>>>> +#define UNICAM_LIP_MASK GENMASK(6, 5)
> >>>>>>>>> +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_IDI0/1 Register */
> >>>>>>>>> +#define UNICAM_ID0_MASK GENMASK(7, 0)
> >>>>>>>>> +#define UNICAM_ID1_MASK GENMASK(15, 8)
> >>>>>>>>> +#define UNICAM_ID2_MASK GENMASK(23, 16)
> >>>>>>>>> +#define UNICAM_ID3_MASK GENMASK(31, 24)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_ISTA Register */
> >>>>>>>>> +#define UNICAM_FSI BIT(0)
> >>>>>>>>> +#define UNICAM_FEI BIT(1)
> >>>>>>>>> +#define UNICAM_LCI BIT(2)
> >>>>>>>>> +
> >>>>>>>>> +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_IPIPE Register */
> >>>>>>>>> +#define UNICAM_PUM_MASK GENMASK(2, 0)
> >>>>>>>>> + /* Unpacking modes */
> >>>>>>>>> + #define UNICAM_PUM_NONE 0
> >>>>>>>>> + #define UNICAM_PUM_UNPACK6 1
> >>>>>>>>> + #define UNICAM_PUM_UNPACK7 2
> >>>>>>>>> + #define UNICAM_PUM_UNPACK8 3
> >>>>>>>>> + #define UNICAM_PUM_UNPACK10 4
> >>>>>>>>> + #define UNICAM_PUM_UNPACK12 5
> >>>>>>>>> + #define UNICAM_PUM_UNPACK14 6
> >>>>>>>>> + #define UNICAM_PUM_UNPACK16 7
> >>>>>>>>> +#define UNICAM_DDM_MASK GENMASK(6, 3)
> >>>>>>>>> +#define UNICAM_PPM_MASK GENMASK(9, 7)
> >>>>>>>>> + /* Packing modes */
> >>>>>>>>> + #define UNICAM_PPM_NONE 0
> >>>>>>>>> + #define UNICAM_PPM_PACK8 1
> >>>>>>>>> + #define UNICAM_PPM_PACK10 2
> >>>>>>>>> + #define UNICAM_PPM_PACK12 3
> >>>>>>>>> + #define UNICAM_PPM_PACK14 4
> >>>>>>>>> + #define UNICAM_PPM_PACK16 5
> >>>>>>>>> +#define UNICAM_DEM_MASK GENMASK(11, 10)
> >>>>>>>>> +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> >>>>>>>>> +#define UNICAM_ICM_MASK GENMASK(16, 15)
> >>>>>>>>> +#define UNICAM_IDM_MASK GENMASK(17, 17)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_ICC Register */
> >>>>>>>>> +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> >>>>>>>>> +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> >>>>>>>>> +#define UNICAM_ICST_MASK GENMASK(12, 10)
> >>>>>>>>> +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> >>>>>>>>> +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_DCS Register */
> >>>>>>>>> +#define UNICAM_DIE BIT(0)
> >>>>>>>>> +#define UNICAM_DIM BIT(1)
> >>>>>>>>> +#define UNICAM_DBOB BIT(3)
> >>>>>>>>> +#define UNICAM_FDE BIT(4)
> >>>>>>>>> +#define UNICAM_LDP BIT(5)
> >>>>>>>>> +#define UNICAM_EDL_MASK GENMASK(15, 8)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_DBCTL Register */
> >>>>>>>>> +#define UNICAM_DBEN BIT(0)
> >>>>>>>>> +#define UNICAM_BUF0_IE BIT(1)
> >>>>>>>>> +#define UNICAM_BUF1_IE BIT(2)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_CMP[0,1] register */
> >>>>>>>>> +#define UNICAM_PCE BIT(31)
> >>>>>>>>> +#define UNICAM_GI BIT(9)
> >>>>>>>>> +#define UNICAM_CPH BIT(8)
> >>>>>>>>> +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> >>>>>>>>> +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> >>>>>>>>> +
> >>>>>>>>> +/* UNICAM_MISC register */
> >>>>>>>>> +#define UNICAM_FL0 BIT(6)
> >>>>>>>>> +#define UNICAM_FL1 BIT(9)
> >>>>>>>>> +
> >>>>>>>>> +#endif
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-05-04 9:25 ` [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface Laurent Pinchart
` (3 preceding siblings ...)
2020-08-29 11:20 ` Jacopo Mondi
@ 2020-09-15 7:03 ` Sakari Ailus
2020-09-15 9:32 ` Laurent Pinchart
2020-09-15 17:30 ` Dave Stevenson
4 siblings, 2 replies; 111+ messages in thread
From: Sakari Ailus @ 2020-09-15 7:03 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Laurent,
Thanks for the patch, and my apologies for the late review. Please do cc me
for v3.
After a quick look I can already say this is the cleanest Unicam driver
I've ever seen. But please also see my detailed comments below.
On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Add a driver for the Unicam camera receiver block on BCM283x processors.
> Compared to the bcm2835-camera driver present in staging, this driver
> handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> the VC4 firmware running on the VPU.
>
> The commit is made up of a series of changes cherry-picked from the
> rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> additional enhancements, forward-ported to the mainline kernel.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
> Changes since v1:
>
> - Re-fetch mbus code from subdev on a g_fmt call
> - Group all ioctl disabling together
> - Fix reference counting in unicam_open
> - Add support for VIDIOC_[S|G]_SELECTION
> ---
> MAINTAINERS | 7 +
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/bcm2835/Kconfig | 15 +
> drivers/media/platform/bcm2835/Makefile | 3 +
> .../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
> .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> 7 files changed, 3106 insertions(+)
> create mode 100644 drivers/media/platform/bcm2835/Kconfig
> create mode 100644 drivers/media/platform/bcm2835/Makefile
> create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index b816a453b10e..edf5b8d9c98a 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3341,6 +3341,13 @@ N: bcm113*
> N: bcm216*
> N: kona
>
> +BROADCOM BCM2835 CAMERA DRIVER
> +M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> +L: linux-media@vger.kernel.org
> +S: Maintained
> +F: drivers/media/platform/bcm2835/
> +F: Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> +
> BROADCOM BCM47XX MIPS ARCHITECTURE
> M: Hauke Mehrtens <hauke@hauke-m.de>
> M: Rafał Miłecki <zajec5@gmail.com>
> diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> index e01bbb9dd1c1..98721a4e0be1 100644
> --- a/drivers/media/platform/Kconfig
> +++ b/drivers/media/platform/Kconfig
> @@ -146,6 +146,7 @@ source "drivers/media/platform/am437x/Kconfig"
> source "drivers/media/platform/xilinx/Kconfig"
> source "drivers/media/platform/rcar-vin/Kconfig"
> source "drivers/media/platform/atmel/Kconfig"
> +source "drivers/media/platform/bcm2835/Kconfig"
> source "drivers/media/platform/sunxi/Kconfig"
>
> config VIDEO_TI_CAL
> diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> index d13db96e3015..a425e4d2e3f3 100644
> --- a/drivers/media/platform/Makefile
> +++ b/drivers/media/platform/Makefile
> @@ -98,4 +98,6 @@ obj-y += meson/
>
> obj-y += cros-ec-cec/
>
> +obj-y += bcm2835/
> +
> obj-y += sunxi/
> diff --git a/drivers/media/platform/bcm2835/Kconfig b/drivers/media/platform/bcm2835/Kconfig
> new file mode 100644
> index 000000000000..ec46e3ef053c
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/Kconfig
> @@ -0,0 +1,15 @@
> +# Broadcom VideoCore4 V4L2 camera support
> +
> +config VIDEO_BCM2835_UNICAM
> + tristate "Broadcom BCM2835 Unicam video capture driver"
> + depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
> + depends on ARCH_BCM2835 || COMPILE_TEST
> + select VIDEOBUF2_DMA_CONTIG
> + select V4L2_FWNODE
> + help
> + Say Y here to enable support for the BCM2835 CSI-2 receiver. This is a
> + V4L2 driver that controls the CSI-2 receiver directly, independently
> + from the VC4 firmware.
\o/
> +
> + To compile this driver as a module, choose M here. The module will be
> + called bcm2835-unicam.
> diff --git a/drivers/media/platform/bcm2835/Makefile b/drivers/media/platform/bcm2835/Makefile
> new file mode 100644
> index 000000000000..a98aba03598a
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/Makefile
> @@ -0,0 +1,3 @@
> +# Makefile for BCM2835 Unicam driver
> +
> +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> new file mode 100644
> index 000000000000..2e9387cbc1e0
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> @@ -0,0 +1,2825 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * BCM2835 Unicam Capture Driver
> + *
> + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> + *
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + *
> + * Based on TI am437x driver by
> + * Benoit Parrot <bparrot@ti.com>
> + * Lad, Prabhakar <prabhakar.csengg@gmail.com>
> + *
> + * and TI CAL camera interface driver by
> + * Benoit Parrot <bparrot@ti.com>
> + *
> + *
> + * There are two camera drivers in the kernel for BCM283x - this one
> + * and bcm2835-camera (currently in staging).
> + *
> + * This driver directly controls the Unicam peripheral - there is no
> + * involvement with the VideoCore firmware. Unicam receives CSI-2 or
> + * CCP2 data and writes it into SDRAM.
> + * The only potential processing options are to repack Bayer data into an
> + * alternate format, and applying windowing.
> + * The repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P
> + * to V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12,
> + * but not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both
> + * formats where the relevant formats are defined, and will automatically
> + * configure the repacking as required.
> + * Support for windowing may be added later.
> + *
> + * It should be possible to connect this driver to any sensor with a
> + * suitable output interface and V4L2 subdevice driver.
> + *
> + * bcm2835-camera uses the VideoCore firmware to control the sensor,
> + * Unicam, ISP, and all tuner control loops. Fully processed frames are
> + * delivered to the driver by the firmware. It only has sensor drivers
> + * for Omnivision OV5647, and Sony IMX219 sensors.
> + *
> + * The two drivers are mutually exclusive for the same Unicam instance.
> + * The VideoCore firmware checks the device tree configuration during boot.
> + * If it finds device tree nodes called csi0 or csi1 it will block the
> + * firmware from accessing the peripheral, and bcm2835-camera will
> + * not be able to stream data.
> + */
> +
> +#include <linux/clk.h>
> +#include <linux/delay.h>
> +#include <linux/device.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/err.h>
> +#include <linux/init.h>
> +#include <linux/interrupt.h>
> +#include <linux/io.h>
> +#include <linux/module.h>
> +#include <linux/of_device.h>
> +#include <linux/of_graph.h>
> +#include <linux/pinctrl/consumer.h>
> +#include <linux/platform_device.h>
> +#include <linux/pm_runtime.h>
> +#include <linux/slab.h>
> +#include <linux/uaccess.h>
> +#include <linux/videodev2.h>
> +
> +#include <media/v4l2-common.h>
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-dev.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-dv-timings.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/v4l2-fwnode.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "vc4-regs-unicam.h"
> +
> +#define UNICAM_MODULE_NAME "unicam"
> +#define UNICAM_VERSION "0.1.0"
> +
> +static int debug;
> +module_param(debug, int, 0644);
> +MODULE_PARM_DESC(debug, "Debug level 0-3");
> +
> +#define unicam_dbg(level, dev, fmt, arg...) \
> + v4l2_dbg(level, debug, &(dev)->v4l2_dev, fmt, ##arg)
> +#define unicam_info(dev, fmt, arg...) \
> + v4l2_info(&(dev)->v4l2_dev, fmt, ##arg)
> +#define unicam_err(dev, fmt, arg...) \
> + v4l2_err(&(dev)->v4l2_dev, fmt, ##arg)
I'd just use dev_*() macros in new drivers unless you really need something
else. It's configurable. Up to you...
> +
> +/*
> + * To protect against a dodgy sensor driver never returning an error from
> + * enum_mbus_code, set a maximum index value to be used.
> + */
> +#define MAX_ENUM_MBUS_CODE 128
Hmm.
Is it Unicam driver's job to work around sensor driver bugs? That same bug
would hit user space, too. I'd say such a sensor driver must be fixed, and
not just worked around here and there.
> +
> +/*
> + * Stride is a 16 bit register, but also has to be a multiple of 32.
> + */
> +#define BPL_ALIGNMENT 32
> +#define MAX_BYTESPERLINE ((1 << 16) - BPL_ALIGNMENT)
> +/*
> + * Max width is therefore determined by the max stride divided by
> + * the number of bits per pixel. Take 32bpp as a
> + * worst case.
> + * No imposed limit on the height, so adopt a square image for want
> + * of anything better.
> + */
> +#define MAX_WIDTH (MAX_BYTESPERLINE / 4)
> +#define MAX_HEIGHT MAX_WIDTH
> +/* Define a nominal minimum image size */
> +#define MIN_WIDTH 16
> +#define MIN_HEIGHT 16
> +/* Default size of the embedded buffer */
> +#define UNICAM_EMBEDDED_SIZE 8192
> +
> +/*
> + * Size of the dummy buffer. Can be any size really, but the DMA
> + * allocation works in units of page sizes.
> + */
> +#define DUMMY_BUF_SIZE (PAGE_SIZE)
> +
> +enum pad_types {
> + IMAGE_PAD,
> + METADATA_PAD,
> + MAX_NODES
> +};
> +
> +/*
> + * struct unicam_fmt - Unicam media bus format information
> + * @pixelformat: V4L2 pixel format FCC identifier. 0 if n/a.
> + * @repacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> + * out to 16bpp. 0 if n/a.
> + * @code: V4L2 media bus format code.
> + * @depth: Bits per pixel as delivered from the source.
> + * @csi_dt: CSI data type.
> + * @check_variants: Flag to denote that there are multiple mediabus formats
> + * still in the list that could match this V4L2 format.
> + */
> +struct unicam_fmt {
> + u32 fourcc;
> + u32 repacked_fourcc;
> + u32 code;
> + u8 depth;
> + u8 csi_dt;
> + u8 check_variants;
> +};
> +
> +static const struct unicam_fmt formats[] = {
> + /* YUV Formats */
> + {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .code = MEDIA_BUS_FMT_YUYV8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .code = MEDIA_BUS_FMT_UYVY8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVYU,
> + .code = MEDIA_BUS_FMT_YVYU8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_VYUY,
> + .code = MEDIA_BUS_FMT_VYUY8_2X8,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + .check_variants = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .code = MEDIA_BUS_FMT_YUYV8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVYU,
> + .code = MEDIA_BUS_FMT_YVYU8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + .fourcc = V4L2_PIX_FMT_VYUY,
> + .code = MEDIA_BUS_FMT_VYUY8_1X16,
> + .depth = 16,
> + .csi_dt = 0x1e,
> + }, {
> + /* RGB Formats */
> + .fourcc = V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
> + .depth = 16,
> + .csi_dt = 0x22,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB565X, /* rrrrrggg gggbbbbb */
> + .code = MEDIA_BUS_FMT_RGB565_2X8_BE,
> + .depth = 16,
> + .csi_dt = 0x22
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB555, /* gggbbbbb arrrrrgg */
> + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
> + .depth = 16,
> + .csi_dt = 0x21,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB555X, /* arrrrrgg gggbbbbb */
> + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
> + .depth = 16,
> + .csi_dt = 0x21,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB24, /* rgb */
> + .code = MEDIA_BUS_FMT_RGB888_1X24,
> + .depth = 24,
> + .csi_dt = 0x24,
> + }, {
> + .fourcc = V4L2_PIX_FMT_BGR24, /* bgr */
> + .code = MEDIA_BUS_FMT_BGR888_1X24,
> + .depth = 24,
> + .csi_dt = 0x24,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB32, /* argb */
> + .code = MEDIA_BUS_FMT_ARGB8888_1X32,
> + .depth = 32,
> + .csi_dt = 0x0,
> + }, {
> + /* Bayer Formats */
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + .code = MEDIA_BUS_FMT_SBGGR8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + .code = MEDIA_BUS_FMT_SGBRG8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + .code = MEDIA_BUS_FMT_SGRBG8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + .code = MEDIA_BUS_FMT_SRGGB8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB10P,
> + .repacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> + .code = MEDIA_BUS_FMT_SBGGR12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> + .code = MEDIA_BUS_FMT_SGBRG12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> + .code = MEDIA_BUS_FMT_SGRBG12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB12P,
> + .repacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> + .code = MEDIA_BUS_FMT_SRGGB12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR14P,
> + .code = MEDIA_BUS_FMT_SBGGR14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG14P,
> + .code = MEDIA_BUS_FMT_SGBRG14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG14P,
> + .code = MEDIA_BUS_FMT_SGRBG14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SRGGB14P,
> + .code = MEDIA_BUS_FMT_SRGGB14_1X14,
> + .depth = 14,
> + .csi_dt = 0x2d,
> + }, {
> + /*
> + * 16 bit Bayer formats could be supported, but there is no CSI2
> + * data_type defined for raw 16, and no sensors that produce it at
> + * present.
> + */
> +
> + /* Greyscale formats */
> + .fourcc = V4L2_PIX_FMT_GREY,
> + .code = MEDIA_BUS_FMT_Y8_1X8,
> + .depth = 8,
> + .csi_dt = 0x2a,
> + }, {
> + .fourcc = V4L2_PIX_FMT_Y10P,
> + .repacked_fourcc = V4L2_PIX_FMT_Y10,
> + .code = MEDIA_BUS_FMT_Y10_1X10,
> + .depth = 10,
> + .csi_dt = 0x2b,
> + }, {
> + /* NB There is no packed V4L2 fourcc for this format. */
> + .repacked_fourcc = V4L2_PIX_FMT_Y12,
> + .code = MEDIA_BUS_FMT_Y12_1X12,
> + .depth = 12,
> + .csi_dt = 0x2c,
> + },
> + /* Embedded data format */
> + {
> + .fourcc = V4L2_META_FMT_SENSOR_DATA,
> + .code = MEDIA_BUS_FMT_SENSOR_DATA,
> + .depth = 8,
> + }
> +};
> +
> +struct unicam_buffer {
> + struct vb2_v4l2_buffer vb;
> + struct list_head list;
> +};
> +
> +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> +{
> + return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> +}
> +
> +struct unicam_node {
> + bool registered;
> + int open;
> + bool streaming;
> + unsigned int pad_id;
> + /* Pointer pointing to current v4l2_buffer */
> + struct unicam_buffer *cur_frm;
> + /* Pointer pointing to next v4l2_buffer */
> + struct unicam_buffer *next_frm;
> + /* video capture */
> + const struct unicam_fmt *fmt;
> + /* Used to store current pixel format */
> + struct v4l2_format v_fmt;
> + /* Used to store current mbus frame format */
> + struct v4l2_mbus_framefmt m_fmt;
> + /* Buffer queue used in video-buf */
> + struct vb2_queue buffer_queue;
> + /* Queue of filled frames */
> + struct list_head dma_queue;
> + /* IRQ lock for DMA queue */
> + spinlock_t dma_queue_lock;
> + /* lock used to access this structure */
> + struct mutex lock;
> + /* Identifies video device for this channel */
> + struct video_device video_dev;
> + /* Pointer to the parent handle */
> + struct unicam_device *dev;
> + struct media_pad pad;
> + unsigned int embedded_lines;
> + /*
> + * Dummy buffer intended to be used by unicam
> + * if we have no other queued buffers to swap to.
> + */
> + void *dummy_buf_cpu_addr;
> + dma_addr_t dummy_buf_dma_addr;
> +};
> +
> +struct unicam_device {
> + struct kref kref;
> +
> + /* V4l2 specific parameters */
> + struct v4l2_async_subdev asd;
> +
> + /* peripheral base address */
> + void __iomem *base;
> + /* clock gating base address */
> + void __iomem *clk_gate_base;
> + /* clock handle */
> + struct clk *clock;
> + /* V4l2 device */
> + struct v4l2_device v4l2_dev;
> + struct media_device mdev;
> +
> + /* parent device */
> + struct platform_device *pdev;
> + /* subdevice async Notifier */
> + struct v4l2_async_notifier notifier;
> + unsigned int sequence;
> +
> + /* ptr to sub device */
> + struct v4l2_subdev *sensor;
> + /* Pad config for the sensor */
> + struct v4l2_subdev_pad_config *sensor_config;
> +
> + enum v4l2_mbus_type bus_type;
> + /*
> + * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> + * bus.mipi_csi1.strobe for CCP2.
> + */
> + unsigned int bus_flags;
> + unsigned int max_data_lanes;
> + unsigned int active_data_lanes;
> + bool sensor_embedded_data;
> +
> + struct unicam_node node[MAX_NODES];
> + struct v4l2_ctrl_handler ctrl_handler;
> +};
> +
> +static inline struct unicam_device *
> +to_unicam_device(struct v4l2_device *v4l2_dev)
> +{
> + return container_of(v4l2_dev, struct unicam_device, v4l2_dev);
> +}
> +
> +/* Hardware access */
> +static inline void clk_write(struct unicam_device *dev, u32 val)
> +{
> + writel(val | 0x5a000000, dev->clk_gate_base);
> +}
> +
> +static inline u32 reg_read(struct unicam_device *dev, u32 offset)
> +{
> + return readl(dev->base + offset);
> +}
> +
> +static inline void reg_write(struct unicam_device *dev, u32 offset, u32 val)
> +{
> + writel(val, dev->base + offset);
> +}
> +
> +static inline int get_field(u32 value, u32 mask)
> +{
> + return (value & mask) >> __ffs(mask);
> +}
> +
> +static inline void set_field(u32 *valp, u32 field, u32 mask)
> +{
> + u32 val = *valp;
> +
> + val &= ~mask;
> + val |= (field << __ffs(mask)) & mask;
> + *valp = val;
> +}
> +
> +static inline u32 reg_read_field(struct unicam_device *dev, u32 offset,
> + u32 mask)
> +{
> + return get_field(reg_read(dev, offset), mask);
> +}
> +
> +static inline void reg_write_field(struct unicam_device *dev, u32 offset,
> + u32 field, u32 mask)
> +{
> + u32 val = reg_read(dev, offset);
> +
> + set_field(&val, field, mask);
> + reg_write(dev, offset, val);
> +}
> +
> +/* Power management functions */
> +static inline int unicam_runtime_get(struct unicam_device *dev)
> +{
> + return pm_runtime_get_sync(&dev->pdev->dev);
> +}
> +
> +static inline void unicam_runtime_put(struct unicam_device *dev)
> +{
> + pm_runtime_put_sync(&dev->pdev->dev);
> +}
Please don't add such wrappers.
> +
> +/* Format setup functions */
> +static const struct unicam_fmt *find_format_by_code(u32 code)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> + if (formats[i].code == code)
> + return &formats[i];
> + }
> +
> + return NULL;
> +}
> +
> +static int check_mbus_format(struct unicam_device *dev,
> + const struct unicam_fmt *format)
> +{
> + unsigned int i;
> + int ret = 0;
> +
> + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = i,
> + .pad = IMAGE_PAD,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
> + NULL, &mbus_code);
> +
> + if (!ret && mbus_code.code == format->code)
> + return 1;
> + }
> +
> + return 0;
> +}
> +
> +static const struct unicam_fmt *find_format_by_pix(struct unicam_device *dev,
> + u32 pixelformat)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> + if (formats[i].fourcc == pixelformat ||
> + formats[i].repacked_fourcc == pixelformat) {
> + if (formats[i].check_variants &&
> + !check_mbus_format(dev, &formats[i]))
> + continue;
> + return &formats[i];
> + }
> + }
> +
> + return NULL;
> +}
> +
> +static inline unsigned int bytes_per_line(u32 width,
> + const struct unicam_fmt *fmt,
> + u32 v4l2_fourcc)
> +{
> + if (v4l2_fourcc == fmt->repacked_fourcc)
> + /* Repacking always goes to 16bpp */
> + return ALIGN(width << 1, BPL_ALIGNMENT);
> + else
> + return ALIGN((width * fmt->depth) >> 3, BPL_ALIGNMENT);
> +}
> +
> +static int __subdev_get_format(struct unicam_device *dev,
> + struct v4l2_mbus_framefmt *fmt, int pad_id)
> +{
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = pad_id
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, get_fmt, dev->sensor_config,
> + &sd_fmt);
> + if (ret < 0)
> + return ret;
> +
> + *fmt = sd_fmt.format;
> +
> + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__,
> + fmt->width, fmt->height, fmt->code);
> +
> + return 0;
> +}
> +
> +static int __subdev_set_format(struct unicam_device *dev,
> + struct v4l2_mbus_framefmt *fmt, int pad_id)
> +{
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = pad_id
> + };
> + int ret;
> +
> + sd_fmt.format = *fmt;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev->sensor_config,
> + &sd_fmt);
> + if (ret < 0)
> + return ret;
> +
> + *fmt = sd_fmt.format;
> +
> + if (pad_id == IMAGE_PAD)
> + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__, fmt->width,
> + fmt->height, fmt->code);
> + else
> + unicam_dbg(1, dev, "%s Embedded data code:%04x\n", __func__,
> + sd_fmt.format.code);
> +
> + return 0;
> +}
> +
> +static int unicam_calc_format_size_bpl(struct unicam_device *dev,
> + const struct unicam_fmt *fmt,
> + struct v4l2_format *f)
> +{
> + unsigned int min_bytesperline;
> +
> + v4l_bound_align_image(&f->fmt.pix.width, MIN_WIDTH, MAX_WIDTH, 2,
> + &f->fmt.pix.height, MIN_HEIGHT, MAX_HEIGHT, 0,
> + 0);
> +
> + min_bytesperline = bytes_per_line(f->fmt.pix.width, fmt,
> + f->fmt.pix.pixelformat);
> +
> + if (f->fmt.pix.bytesperline > min_bytesperline &&
> + f->fmt.pix.bytesperline <= MAX_BYTESPERLINE)
> + f->fmt.pix.bytesperline = ALIGN(f->fmt.pix.bytesperline,
> + BPL_ALIGNMENT);
> + else
> + f->fmt.pix.bytesperline = min_bytesperline;
> +
> + f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
> +
> + unicam_dbg(3, dev, "%s: fourcc: %08X size: %dx%d bpl:%d img_size:%d\n",
> + __func__,
> + f->fmt.pix.pixelformat,
> + f->fmt.pix.width, f->fmt.pix.height,
> + f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
> +
> + return 0;
> +}
> +
> +static int unicam_reset_format(struct unicam_node *node)
> +{
> + struct unicam_device *dev = node->dev;
> + struct v4l2_mbus_framefmt mbus_fmt;
> + int ret;
> +
> + if (dev->sensor_embedded_data || node->pad_id != METADATA_PAD) {
> + ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
> + if (ret) {
> + unicam_err(dev, "Failed to get_format - ret %d\n", ret);
> + return ret;
> + }
> +
> + if (mbus_fmt.code != node->fmt->code) {
> + unicam_err(dev, "code mismatch - fmt->code %08x, mbus_fmt.code %08x\n",
> + node->fmt->code, mbus_fmt.code);
> + return ret;
> + }
> + }
> +
> + if (node->pad_id == IMAGE_PAD) {
> + v4l2_fill_pix_format(&node->v_fmt.fmt.pix, &mbus_fmt);
> + node->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + unicam_calc_format_size_bpl(dev, node->fmt, &node->v_fmt);
> + } else {
> + node->v_fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> + node->v_fmt.fmt.meta.dataformat = V4L2_META_FMT_SENSOR_DATA;
> + if (dev->sensor_embedded_data) {
> + node->v_fmt.fmt.meta.buffersize =
> + mbus_fmt.width * mbus_fmt.height;
> + node->embedded_lines = mbus_fmt.height;
> + } else {
> + node->v_fmt.fmt.meta.buffersize = UNICAM_EMBEDDED_SIZE;
> + node->embedded_lines = 1;
> + }
> + }
> +
> + node->m_fmt = mbus_fmt;
> + return 0;
> +}
> +
> +static void unicam_wr_dma_addr(struct unicam_device *dev, dma_addr_t dmaaddr,
> + unsigned int buffer_size, int pad_id)
> +{
> + dma_addr_t endaddr = dmaaddr + buffer_size;
> +
> + /*
> + * dmaaddr and endaddr should be a 32-bit address with the top two bits
> + * set to 0x3 to signify uncached access through the Videocore memory
> + * controller.
> + */
> + WARN_ON((dmaaddr >> 30) != 0x3 || (endaddr >> 30) != 0x3);
> +
> + if (pad_id == IMAGE_PAD) {
> + reg_write(dev, UNICAM_IBSA0, dmaaddr);
> + reg_write(dev, UNICAM_IBEA0, endaddr);
> + } else {
> + reg_write(dev, UNICAM_DBSA0, dmaaddr);
> + reg_write(dev, UNICAM_DBEA0, endaddr);
> + }
> +}
> +
> +static inline unsigned int unicam_get_lines_done(struct unicam_device *dev)
> +{
> + dma_addr_t start_addr, cur_addr;
> + unsigned int stride = dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline;
> + struct unicam_buffer *frm = dev->node[IMAGE_PAD].cur_frm;
> +
> + if (!frm)
> + return 0;
> +
> + start_addr = vb2_dma_contig_plane_dma_addr(&frm->vb.vb2_buf, 0);
> + cur_addr = reg_read(dev, UNICAM_IBWP);
> + return (unsigned int)(cur_addr - start_addr) / stride;
> +}
> +
> +static inline void unicam_schedule_next_buffer(struct unicam_node *node)
> +{
> + struct unicam_device *dev = node->dev;
> + struct unicam_buffer *buf;
> + unsigned int size;
> + dma_addr_t addr;
> +
> + buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> + node->next_frm = buf;
> + list_del(&buf->list);
> +
> + addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> + size = (node->pad_id == IMAGE_PAD) ?
> + node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> +
> + unicam_wr_dma_addr(dev, addr, size, node->pad_id);
> +}
> +
> +static inline void unicam_schedule_dummy_buffer(struct unicam_node *node)
> +{
> + struct unicam_device *dev = node->dev;
> +
> + unicam_dbg(3, dev, "Scheduling dummy buffer for node %d\n",
> + node->pad_id);
> +
> + unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr, DUMMY_BUF_SIZE,
> + node->pad_id);
> + node->next_frm = NULL;
> +}
> +
> +static inline void unicam_process_buffer_complete(struct unicam_node *node,
> + unsigned int sequence)
> +{
> + node->cur_frm->vb.field = node->m_fmt.field;
> + node->cur_frm->vb.sequence = sequence;
> +
> + vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> +}
> +
> +static bool unicam_all_nodes_streaming(struct unicam_device *dev)
> +{
> + bool ret;
> +
> + ret = dev->node[IMAGE_PAD].open && dev->node[IMAGE_PAD].streaming;
> + ret &= !dev->node[METADATA_PAD].open ||
> + dev->node[METADATA_PAD].streaming;
> + return ret;
> +}
> +
> +static bool unicam_all_nodes_disabled(struct unicam_device *dev)
> +{
> + return !dev->node[IMAGE_PAD].streaming &&
> + !dev->node[METADATA_PAD].streaming;
> +}
> +
> +static void unicam_queue_event_sof(struct unicam_device *unicam)
> +{
> + struct v4l2_event event = {
> + .type = V4L2_EVENT_FRAME_SYNC,
> + .u.frame_sync.frame_sequence = unicam->sequence,
> + };
> +
> + v4l2_event_queue(&unicam->node[IMAGE_PAD].video_dev, &event);
> +}
> +
> +/*
> + * unicam_isr : ISR handler for unicam capture
> + * @irq: irq number
> + * @dev_id: dev_id ptr
> + *
> + * It changes status of the captured buffer, takes next buffer from the queue
> + * and sets its address in unicam registers
> + */
> +static irqreturn_t unicam_isr(int irq, void *dev)
> +{
> + struct unicam_device *unicam = dev;
> + unsigned int lines_done = unicam_get_lines_done(dev);
> + unsigned int sequence = unicam->sequence;
> + unsigned int i;
> + u32 ista, sta;
> + u64 ts;
> +
> + /*
> + * Don't service interrupts if not streaming.
> + * Avoids issues if the VPU should enable the
> + * peripheral without the kernel knowing (that
> + * shouldn't happen, but causes issues if it does).
> + */
> + if (unicam_all_nodes_disabled(unicam))
> + return IRQ_NONE;
> +
> + sta = reg_read(unicam, UNICAM_STA);
> + /* Write value back to clear the interrupts */
> + reg_write(unicam, UNICAM_STA, sta);
> +
> + ista = reg_read(unicam, UNICAM_ISTA);
> + /* Write value back to clear the interrupts */
> + reg_write(unicam, UNICAM_ISTA, ista);
> +
> + unicam_dbg(3, unicam, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d",
> + ista, sta, sequence, lines_done);
> +
> + if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> + return IRQ_HANDLED;
> +
> + /*
> + * We must run the frame end handler first. If we have a valid next_frm
> + * and we get a simultaneout FE + FS interrupt, running the FS handler
> + * first would null out the next_frm ptr and we would have lost the
> + * buffer forever.
> + */
> + if (ista & UNICAM_FEI || sta & UNICAM_PI0) {
> + /*
> + * Ensure we have swapped buffers already as we can't
> + * stop the peripheral. If no buffer is available, use a
> + * dummy buffer to dump out frames until we get a new buffer
> + * to use.
> + */
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + if (unicam->node[i].cur_frm)
> + unicam_process_buffer_complete(&unicam->node[i],
> + sequence);
> + unicam->node[i].cur_frm = unicam->node[i].next_frm;
> + }
> + unicam->sequence++;
> + }
> +
> + if (ista & UNICAM_FSI) {
> + /*
> + * Timestamp is to be when the first data byte was captured,
> + * aka frame start.
> + */
> + ts = ktime_get_ns();
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + if (unicam->node[i].cur_frm)
> + unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> + ts;
> + /*
> + * Set the next frame output to go to a dummy frame
> + * if we have not managed to obtain another frame
> + * from the queue.
> + */
> + unicam_schedule_dummy_buffer(&unicam->node[i]);
> + }
> +
> + unicam_queue_event_sof(unicam);
> + }
> +
> + /*
> + * Cannot swap buffer at frame end, there may be a race condition
> + * where the HW does not actually swap it if the new frame has
> + * already started.
> + */
> + if (ista & (UNICAM_FSI | UNICAM_LCI) && !(ista & UNICAM_FEI)) {
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + if (!unicam->node[i].streaming)
> + continue;
> +
> + spin_lock(&unicam->node[i].dma_queue_lock);
> + if (!list_empty(&unicam->node[i].dma_queue) &&
> + !unicam->node[i].next_frm)
> + unicam_schedule_next_buffer(&unicam->node[i]);
> + spin_unlock(&unicam->node[i].dma_queue_lock);
> + }
> + }
> +
> + if (reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> + /* Switch out of trigger mode if selected */
> + reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> + reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> + }
> + return IRQ_HANDLED;
> +}
> +
> +static int unicam_querycap(struct file *file, void *priv,
> + struct v4l2_capability *cap)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + strlcpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> + strlcpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
strscpy()?
> +
> + snprintf(cap->bus_info, sizeof(cap->bus_info),
> + "platform:%s", dev_name(&dev->pdev->dev));
> +
> + cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
Please also fill cap->device_caps.
> +
> + return 0;
> +}
> +
> +static int unicam_enum_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_fmtdesc *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + unsigned int index = 0;
> + unsigned int i;
> + int ret = 0;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = i,
> + .pad = IMAGE_PAD,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> + const struct unicam_fmt *fmt;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
> + NULL, &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx %d returned %d - index invalid\n",
> + i, ret);
> + return -EINVAL;
> + }
> +
> + fmt = find_format_by_code(mbus_code.code);
> + if (fmt) {
> + if (fmt->fourcc) {
> + if (index == f->index) {
> + f->pixelformat = fmt->fourcc;
> + break;
> + }
> + index++;
> + }
> + if (fmt->repacked_fourcc) {
> + if (index == f->index) {
> + f->pixelformat = fmt->repacked_fourcc;
> + break;
> + }
> + index++;
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> +static int unicam_g_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt = NULL;
> + int ret;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + /*
> + * If a flip has occurred in the sensor, the fmt code might have
> + * changed. So we will need to re-fetch the format from the subdevice.
> + */
> + ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
> + if (ret)
> + return -EINVAL;
> +
> + /* Find the V4L2 format from mbus code. We must match a known format. */
> + fmt = find_format_by_code(mbus_fmt.code);
> + if (!fmt)
> + return -EINVAL;
> +
> + node->fmt = fmt;
> + node->v_fmt.fmt.pix.pixelformat = fmt->fourcc;
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static
> +const struct unicam_fmt *get_first_supported_format(struct unicam_device *dev)
> +{
> + struct v4l2_subdev_mbus_code_enum mbus_code;
> + const struct unicam_fmt *fmt = NULL;
> + unsigned int i;
> + int ret;
> +
> + for (i = 0; ret != -EINVAL && ret != -ENOIOCTLCMD; ++i) {
> + memset(&mbus_code, 0, sizeof(mbus_code));
> + mbus_code.index = i;
> + mbus_code.pad = IMAGE_PAD;
> + mbus_code.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code, NULL,
> + &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx %u returned %d - continue\n",
> + i, ret);
> + continue;
> + }
> +
> + unicam_dbg(2, dev, "subdev %s: code: 0x%08x idx: %u\n",
> + dev->sensor->name, mbus_code.code, i);
> +
> + fmt = find_format_by_code(mbus_code.code);
> + unicam_dbg(2, dev, "fmt 0x%08x returned as %p, V4L2 FOURCC 0x%08x, csi_dt 0x%02x\n",
> + mbus_code.code, fmt, fmt ? fmt->fourcc : 0,
> + fmt ? fmt->csi_dt : 0);
> + if (fmt)
> + return fmt;
> + }
> +
> + return NULL;
> +}
> +
> +static int unicam_try_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_format sd_fmt = {
> + .which = V4L2_SUBDEV_FORMAT_TRY,
> + .pad = IMAGE_PAD
> + };
> + struct v4l2_mbus_framefmt *mbus_fmt = &sd_fmt.format;
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (node->pad_id != IMAGE_PAD)
> + return -EINVAL;
> +
> + fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
> + if (!fmt) {
> + /*
> + * Pixel format not supported by unicam. Choose the first
> + * supported format, and let the sensor choose something else.
> + */
> + unicam_dbg(3, dev, "Fourcc format (0x%08x) not found. Use first format.\n",
> + f->fmt.pix.pixelformat);
> +
> + fmt = &formats[0];
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + }
> +
> + v4l2_fill_mbus_format(mbus_fmt, &f->fmt.pix, fmt->code);
> + /*
> + * No support for receiving interlaced video, so never
> + * request it from the sensor subdev.
> + */
> + mbus_fmt->field = V4L2_FIELD_NONE;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev->sensor_config,
> + &sd_fmt);
> + if (ret && ret != -ENOIOCTLCMD && ret != -ENODEV)
> + return ret;
> +
> + if (mbus_fmt->field != V4L2_FIELD_NONE)
> + unicam_info(dev, "Sensor trying to send interlaced video - results may be unpredictable\n");
> +
> + v4l2_fill_pix_format(&f->fmt.pix, &sd_fmt.format);
> + if (mbus_fmt->code != fmt->code) {
> + /* Sensor has returned an alternate format */
> + fmt = find_format_by_code(mbus_fmt->code);
> + if (!fmt) {
> + /*
> + * The alternate format is one unicam can't support.
> + * Find the first format that is supported by both, and
> + * then set that.
> + */
> + fmt = get_first_supported_format(dev);
> + mbus_fmt->code = fmt->code;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt,
> + dev->sensor_config, &sd_fmt);
> + if (ret && ret != -ENOIOCTLCMD && ret != -ENODEV)
> + return ret;
> +
> + if (mbus_fmt->field != V4L2_FIELD_NONE)
> + unicam_info(dev, "Sensor trying to send interlaced video - results may be unpredictable\n");
> +
> + v4l2_fill_pix_format(&f->fmt.pix, &sd_fmt.format);
> +
> + if (mbus_fmt->code != fmt->code) {
> + /*
> + * We've set a format that the sensor reports
> + * as being supported, but it refuses to set it.
> + * Not much else we can do.
> + * Assume that the sensor driver may accept the
> + * format when it is set (rather than tried).
> + */
> + unicam_err(dev, "Sensor won't accept default format, and Unicam can't support sensor default\n");
> + }
> + }
> +
> + if (fmt->fourcc)
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + else
> + f->fmt.pix.pixelformat = fmt->repacked_fourcc;
> + }
> +
> + return unicam_calc_format_size_bpl(dev, fmt, f);
> +}
> +
> +static int unicam_s_fmt_vid_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct vb2_queue *q = &node->buffer_queue;
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (vb2_is_busy(q))
> + return -EBUSY;
> +
> + ret = unicam_try_fmt_vid_cap(file, priv, f);
> + if (ret < 0)
> + return ret;
> +
> + fmt = find_format_by_pix(dev, f->fmt.pix.pixelformat);
> + if (!fmt) {
> + /*
> + * Unknown pixel format - adopt a default.
> + * This shouldn't happen as try_fmt should have resolved any
> + * issues first.
> + */
> + fmt = get_first_supported_format(dev);
> + if (!fmt)
> + /*
> + * It shouldn't be possible to get here with no
> + * supported formats
> + */
> + return -EINVAL;
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + return -EINVAL;
> + }
> +
> + v4l2_fill_mbus_format(&mbus_fmt, &f->fmt.pix, fmt->code);
> +
> + ret = __subdev_set_format(dev, &mbus_fmt, node->pad_id);
> + if (ret) {
> + unicam_dbg(3, dev, "%s __subdev_set_format failed %d\n",
> + __func__, ret);
> + return ret;
> + }
> +
> + /* Just double check nothing has gone wrong */
> + if (mbus_fmt.code != fmt->code) {
> + unicam_dbg(3, dev,
> + "%s subdev changed format on us, this should not happen\n",
> + __func__);
> + return -EINVAL;
> + }
> +
> + node->fmt = fmt;
> + node->v_fmt.fmt.pix.pixelformat = f->fmt.pix.pixelformat;
> + node->v_fmt.fmt.pix.bytesperline = f->fmt.pix.bytesperline;
> + unicam_reset_format(node);
> +
> + unicam_dbg(3, dev,
> + "%s %dx%d, mbus_fmt 0x%08X, V4L2 pix 0x%08X.\n",
> + __func__, node->v_fmt.fmt.pix.width,
> + node->v_fmt.fmt.pix.height, mbus_fmt.code,
> + node->v_fmt.fmt.pix.pixelformat);
> +
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_fmt_meta_cap(struct file *file, void *priv,
> + struct v4l2_fmtdesc *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + u32 code;
> + int ret = 0;
> +
> + if (node->pad_id != METADATA_PAD || f->index != 0)
> + return -EINVAL;
> +
> + if (dev->sensor_embedded_data) {
> + struct v4l2_subdev_mbus_code_enum mbus_code = {
> + .index = f->index,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .pad = METADATA_PAD,
> + };
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code, NULL,
> + &mbus_code);
> + if (ret < 0) {
> + unicam_dbg(2, dev,
> + "subdev->enum_mbus_code idx 0 returned %d - index invalid\n",
> + ret);
> + return -EINVAL;
> + }
> +
> + code = mbus_code.code;
> + } else {
> + code = MEDIA_BUS_FMT_SENSOR_DATA;
> + }
> +
> + fmt = find_format_by_code(code);
> + if (fmt)
> + f->pixelformat = fmt->fourcc;
> +
> + return 0;
> +}
> +
> +static int unicam_g_fmt_meta_cap(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct unicam_node *node = video_drvdata(file);
> +
> + if (node->pad_id != METADATA_PAD)
> + return -EINVAL;
> +
> + *f = node->v_fmt;
> +
> + return 0;
> +}
> +
> +static int unicam_queue_setup(struct vb2_queue *vq,
> + unsigned int *nbuffers,
> + unsigned int *nplanes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> + unsigned int size = node->pad_id == IMAGE_PAD ?
> + node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> +
> + if (vq->num_buffers + *nbuffers < 3)
> + *nbuffers = 3 - vq->num_buffers;
> +
> + if (*nplanes) {
> + if (sizes[0] < size) {
> + unicam_err(dev, "sizes[0] %i < size %u\n", sizes[0],
> + size);
> + return -EINVAL;
> + }
> + size = sizes[0];
> + }
> +
> + *nplanes = 1;
> + sizes[0] = size;
> +
> + return 0;
> +}
> +
> +static int unicam_buffer_prepare(struct vb2_buffer *vb)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct unicam_device *dev = node->dev;
> + struct unicam_buffer *buf = to_unicam_buffer(vb);
> + unsigned long size;
> +
> + if (WARN_ON(!node->fmt))
> + return -EINVAL;
> +
> + size = node->pad_id == IMAGE_PAD ? node->v_fmt.fmt.pix.sizeimage :
> + node->v_fmt.fmt.meta.buffersize;
> + if (vb2_plane_size(vb, 0) < size) {
> + unicam_err(dev, "data will not fit into plane (%lu < %lu)\n",
> + vb2_plane_size(vb, 0), size);
> + return -EINVAL;
> + }
> +
> + vb2_set_plane_payload(&buf->vb.vb2_buf, 0, size);
> + return 0;
> +}
> +
> +static void unicam_buffer_queue(struct vb2_buffer *vb)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct unicam_buffer *buf = to_unicam_buffer(vb);
> + unsigned long flags;
> +
> + spin_lock_irqsave(&node->dma_queue_lock, flags);
> + list_add_tail(&buf->list, &node->dma_queue);
> + spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +}
> +
> +static void unicam_set_packing_config(struct unicam_device *dev)
> +{
> + u32 pack, unpack;
> + u32 val;
> +
> + if (dev->node[IMAGE_PAD].v_fmt.fmt.pix.pixelformat ==
> + dev->node[IMAGE_PAD].fmt->fourcc) {
> + unpack = UNICAM_PUM_NONE;
> + pack = UNICAM_PPM_NONE;
> + } else {
> + switch (dev->node[IMAGE_PAD].fmt->depth) {
> + case 8:
> + unpack = UNICAM_PUM_UNPACK8;
> + break;
> + case 10:
> + unpack = UNICAM_PUM_UNPACK10;
> + break;
> + case 12:
> + unpack = UNICAM_PUM_UNPACK12;
> + break;
> + case 14:
> + unpack = UNICAM_PUM_UNPACK14;
> + break;
> + case 16:
> + unpack = UNICAM_PUM_UNPACK16;
> + break;
> + default:
> + unpack = UNICAM_PUM_NONE;
> + break;
> + }
> +
> + /* Repacking is always to 16bpp */
> + pack = UNICAM_PPM_PACK16;
> + }
> +
> + val = 0;
> + set_field(&val, unpack, UNICAM_PUM_MASK);
> + set_field(&val, pack, UNICAM_PPM_MASK);
> + reg_write(dev, UNICAM_IPIPE, val);
> +}
> +
> +static void unicam_cfg_image_id(struct unicam_device *dev)
> +{
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 mode, hardcode VC 0 for now. */
> + reg_write(dev, UNICAM_IDI0,
> + (0 << 6) | dev->node[IMAGE_PAD].fmt->csi_dt);
> + } else {
> + /* CCP2 mode */
> + reg_write(dev, UNICAM_IDI0,
> + 0x80 | dev->node[IMAGE_PAD].fmt->csi_dt);
> + }
> +}
> +
> +static void unicam_enable_ed(struct unicam_device *dev)
> +{
> + u32 val = reg_read(dev, UNICAM_DCS);
> +
> + set_field(&val, 2, UNICAM_EDL_MASK);
> + /* Do not wrap at the end of the embedded data buffer */
> + set_field(&val, 0, UNICAM_DBOB);
> +
> + reg_write(dev, UNICAM_DCS, val);
> +}
> +
> +static void unicam_start_rx(struct unicam_device *dev, dma_addr_t *addr)
> +{
> + int line_int_freq = dev->node[IMAGE_PAD].v_fmt.fmt.pix.height >> 2;
> + unsigned int size, i;
> + u32 val;
> +
> + if (line_int_freq < 128)
> + line_int_freq = 128;
> +
> + /* Enable lane clocks */
> + val = 1;
> + for (i = 0; i < dev->active_data_lanes; i++)
> + val = val << 2 | 1;
> + clk_write(dev, val);
> +
> + /* Basic init */
> + reg_write(dev, UNICAM_CTRL, UNICAM_MEM);
> +
> + /* Enable analogue control, and leave in reset. */
> + val = UNICAM_AR;
> + set_field(&val, 7, UNICAM_CTATADJ_MASK);
> + set_field(&val, 7, UNICAM_PTATADJ_MASK);
> + reg_write(dev, UNICAM_ANA, val);
> + usleep_range(1000, 2000);
> +
> + /* Come out of reset */
> + reg_write_field(dev, UNICAM_ANA, 0, UNICAM_AR);
> +
> + /* Peripheral reset */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> + /* Enable Rx control. */
> + val = reg_read(dev, UNICAM_CTRL);
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + set_field(&val, UNICAM_CPM_CSI2, UNICAM_CPM_MASK);
> + set_field(&val, UNICAM_DCM_STROBE, UNICAM_DCM_MASK);
> + } else {
> + set_field(&val, UNICAM_CPM_CCP2, UNICAM_CPM_MASK);
> + set_field(&val, dev->bus_flags, UNICAM_DCM_MASK);
> + }
> + /* Packet framer timeout */
> + set_field(&val, 0xf, UNICAM_PFT_MASK);
> + set_field(&val, 128, UNICAM_OET_MASK);
> + reg_write(dev, UNICAM_CTRL, val);
> +
> + reg_write(dev, UNICAM_IHWIN, 0);
> + reg_write(dev, UNICAM_IVWIN, 0);
> +
> + /* AXI bus access QoS setup */
> + val = reg_read(dev, UNICAM_PRI);
> + set_field(&val, 0, UNICAM_BL_MASK);
> + set_field(&val, 0, UNICAM_BS_MASK);
> + set_field(&val, 0xe, UNICAM_PP_MASK);
> + set_field(&val, 8, UNICAM_NP_MASK);
> + set_field(&val, 2, UNICAM_PT_MASK);
> + set_field(&val, 1, UNICAM_PE);
> + reg_write(dev, UNICAM_PRI, val);
> +
> + reg_write_field(dev, UNICAM_ANA, 0, UNICAM_DDL);
> +
> + /* Always start in trigger frame capture mode (UNICAM_FCM set) */
> + val = UNICAM_FSIE | UNICAM_FEIE | UNICAM_FCM | UNICAM_IBOB;
> + set_field(&val, line_int_freq, UNICAM_LCIE_MASK);
> + reg_write(dev, UNICAM_ICTL, val);
> + reg_write(dev, UNICAM_STA, UNICAM_STA_MASK_ALL);
> + reg_write(dev, UNICAM_ISTA, UNICAM_ISTA_MASK_ALL);
> +
> + /* tclk_term_en */
> + reg_write_field(dev, UNICAM_CLT, 2, UNICAM_CLT1_MASK);
> + /* tclk_settle */
> + reg_write_field(dev, UNICAM_CLT, 6, UNICAM_CLT2_MASK);
> + /* td_term_en */
> + reg_write_field(dev, UNICAM_DLT, 2, UNICAM_DLT1_MASK);
> + /* ths_settle */
> + reg_write_field(dev, UNICAM_DLT, 6, UNICAM_DLT2_MASK);
> + /* trx_enable */
> + reg_write_field(dev, UNICAM_DLT, 0, UNICAM_DLT3_MASK);
> +
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_SOE);
> +
> + /* Packet compare setup - required to avoid missing frame ends */
> + val = 0;
> + set_field(&val, 1, UNICAM_PCE);
> + set_field(&val, 1, UNICAM_GI);
> + set_field(&val, 1, UNICAM_CPH);
> + set_field(&val, 0, UNICAM_PCVC_MASK);
> + set_field(&val, 1, UNICAM_PCDT_MASK);
> + reg_write(dev, UNICAM_CMP0, val);
> +
> + /* Enable clock lane and set up terminations */
> + val = 0;
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 */
> + set_field(&val, 1, UNICAM_CLE);
> + set_field(&val, 1, UNICAM_CLLPE);
> + if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
> + set_field(&val, 1, UNICAM_CLTRE);
> + set_field(&val, 1, UNICAM_CLHSE);
> + }
> + } else {
> + /* CCP2 */
> + set_field(&val, 1, UNICAM_CLE);
> + set_field(&val, 1, UNICAM_CLHSE);
> + set_field(&val, 1, UNICAM_CLTRE);
> + }
> + reg_write(dev, UNICAM_CLK, val);
> +
> + /*
> + * Enable required data lanes with appropriate terminations.
> + * The same value needs to be written to UNICAM_DATn registers for
> + * the active lanes, and 0 for inactive ones.
> + */
> + val = 0;
> + if (dev->bus_type == V4L2_MBUS_CSI2_DPHY) {
> + /* CSI2 */
> + set_field(&val, 1, UNICAM_DLE);
> + set_field(&val, 1, UNICAM_DLLPE);
> + if (dev->bus_flags & V4L2_MBUS_CSI2_CONTINUOUS_CLOCK) {
> + set_field(&val, 1, UNICAM_DLTRE);
> + set_field(&val, 1, UNICAM_DLHSE);
> + }
> + } else {
> + /* CCP2 */
> + set_field(&val, 1, UNICAM_DLE);
> + set_field(&val, 1, UNICAM_DLHSE);
> + set_field(&val, 1, UNICAM_DLTRE);
> + }
> + reg_write(dev, UNICAM_DAT0, val);
> +
> + if (dev->active_data_lanes == 1)
> + val = 0;
> + reg_write(dev, UNICAM_DAT1, val);
> +
> + if (dev->max_data_lanes > 2) {
> + /*
> + * Registers UNICAM_DAT2 and UNICAM_DAT3 only valid if the
> + * instance supports more than 2 data lanes.
> + */
> + if (dev->active_data_lanes == 2)
> + val = 0;
> + reg_write(dev, UNICAM_DAT2, val);
> +
> + if (dev->active_data_lanes == 3)
> + val = 0;
> + reg_write(dev, UNICAM_DAT3, val);
> + }
> +
> + reg_write(dev, UNICAM_IBLS,
> + dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline);
> + size = dev->node[IMAGE_PAD].v_fmt.fmt.pix.sizeimage;
> + unicam_wr_dma_addr(dev, addr[IMAGE_PAD], size, IMAGE_PAD);
> + unicam_set_packing_config(dev);
> + unicam_cfg_image_id(dev);
> +
> + val = reg_read(dev, UNICAM_MISC);
> + set_field(&val, 1, UNICAM_FL0);
> + set_field(&val, 1, UNICAM_FL1);
> + reg_write(dev, UNICAM_MISC, val);
> +
> + if (dev->node[METADATA_PAD].streaming && dev->sensor_embedded_data) {
> + size = dev->node[METADATA_PAD].v_fmt.fmt.meta.buffersize;
> + unicam_enable_ed(dev);
> + unicam_wr_dma_addr(dev, addr[METADATA_PAD], size, METADATA_PAD);
> + }
> +
> + /* Enable peripheral */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPE);
> +
> + /* Load image pointers */
> + reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_LIP_MASK);
> +
> + /* Load embedded data buffer pointers if needed */
> + if (dev->node[METADATA_PAD].streaming && dev->sensor_embedded_data)
> + reg_write_field(dev, UNICAM_DCS, 1, UNICAM_LDP);
> +
> + /*
> + * Enable trigger only for the first frame to
> + * sync correctly to the FS from the source.
> + */
> + reg_write_field(dev, UNICAM_ICTL, 1, UNICAM_TFC);
> +}
> +
> +static void unicam_disable(struct unicam_device *dev)
> +{
> + /* Analogue lane control disable */
> + reg_write_field(dev, UNICAM_ANA, 1, UNICAM_DDL);
> +
> + /* Stop the output engine */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_SOE);
> +
> + /* Disable the data lanes. */
> + reg_write(dev, UNICAM_DAT0, 0);
> + reg_write(dev, UNICAM_DAT1, 0);
> +
> + if (dev->max_data_lanes > 2) {
> + reg_write(dev, UNICAM_DAT2, 0);
> + reg_write(dev, UNICAM_DAT3, 0);
> + }
> +
> + /* Peripheral reset */
> + reg_write_field(dev, UNICAM_CTRL, 1, UNICAM_CPR);
> + usleep_range(50, 100);
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPR);
> +
> + /* Disable peripheral */
> + reg_write_field(dev, UNICAM_CTRL, 0, UNICAM_CPE);
> +
> + /* Clear ED setup */
> + reg_write(dev, UNICAM_DCS, 0);
> +
> + /* Disable all lane clocks */
> + clk_write(dev, 0);
> +}
> +
> +static void unicam_return_buffers(struct unicam_node *node)
> +{
> + struct unicam_buffer *buf, *tmp;
> + unsigned long flags;
> +
> + spin_lock_irqsave(&node->dma_queue_lock, flags);
> + list_for_each_entry_safe(buf, tmp, &node->dma_queue, list) {
> + list_del(&buf->list);
> + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
> + }
> +
> + if (node->cur_frm)
> + vb2_buffer_done(&node->cur_frm->vb.vb2_buf,
> + VB2_BUF_STATE_ERROR);
> + if (node->next_frm && node->cur_frm != node->next_frm)
> + vb2_buffer_done(&node->next_frm->vb.vb2_buf,
> + VB2_BUF_STATE_ERROR);
> +
> + node->cur_frm = NULL;
> + node->next_frm = NULL;
> + spin_unlock_irqrestore(&node->dma_queue_lock, flags);
> +}
> +
> +static int unicam_start_streaming(struct vb2_queue *vq, unsigned int count)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> + dma_addr_t buffer_addr[MAX_NODES] = { 0 };
> + unsigned long flags;
> + unsigned int i;
> + int ret;
> +
> + node->streaming = true;
> + if (!unicam_all_nodes_streaming(dev)) {
> + unicam_dbg(3, dev, "Not all nodes are streaming yet.");
> + return 0;
> + }
> +
> + dev->sequence = 0;
> + ret = unicam_runtime_get(dev);
> + if (ret < 0) {
> + unicam_dbg(3, dev, "unicam_runtime_get failed\n");
> + goto err_streaming;
> + }
> +
> + /*
> + * TODO: Retrieve the number of active data lanes from the connected
> + * subdevice.
> + */
> + dev->active_data_lanes = dev->max_data_lanes;
> +
> + ret = clk_set_rate(dev->clock, 100 * 1000 * 1000);
> + if (ret) {
> + unicam_err(dev, "failed to set up clock\n");
> + goto err_pm_put;
> + }
> +
> + ret = clk_prepare_enable(dev->clock);
> + if (ret) {
> + unicam_err(dev, "Failed to enable CSI clock: %d\n", ret);
> + goto err_pm_put;
> + }
> +
> + for (i = 0; i < ARRAY_SIZE(dev->node); i++) {
> + struct unicam_buffer *buf;
> +
> + if (!dev->node[i].streaming)
> + continue;
> +
> + spin_lock_irqsave(&dev->node[i].dma_queue_lock, flags);
> + buf = list_first_entry(&dev->node[i].dma_queue,
> + struct unicam_buffer, list);
> + dev->node[i].cur_frm = buf;
> + dev->node[i].next_frm = buf;
> + list_del(&buf->list);
> + spin_unlock_irqrestore(&dev->node[i].dma_queue_lock, flags);
> +
> + buffer_addr[i] =
> + vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> + }
> +
> + unicam_start_rx(dev, buffer_addr);
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_stream, 1);
> + if (ret < 0) {
> + unicam_err(dev, "stream on failed in subdev\n");
> + goto err_disable_unicam;
> + }
> +
> + return 0;
> +
> +err_disable_unicam:
> + unicam_disable(dev);
> + clk_disable_unprepare(dev->clock);
> +err_pm_put:
> + unicam_runtime_put(dev);
> +err_streaming:
> + unicam_return_buffers(node);
> + node->streaming = false;
> +
> + return ret;
> +}
> +
> +static void unicam_stop_streaming(struct vb2_queue *vq)
> +{
> + struct unicam_node *node = vb2_get_drv_priv(vq);
> + struct unicam_device *dev = node->dev;
> +
> + node->streaming = false;
> +
> + if (node->pad_id == IMAGE_PAD) {
> + /*
> + * Stop streaming the sensor and disable the peripheral.
> + * We cannot continue streaming embedded data with the
> + * image pad disabled.
> + */
> + if (v4l2_subdev_call(dev->sensor, video, s_stream, 0) < 0)
> + unicam_err(dev, "stream off failed in subdev\n");
> +
> + unicam_disable(dev);
> + clk_disable_unprepare(dev->clock);
> + unicam_runtime_put(dev);
> +
> + } else if (node->pad_id == METADATA_PAD) {
> + /*
> + * Allow the hardware to spin in the dummy buffer.
> + * This is only really needed if the embedded data pad is
> + * disabled before the image pad.
> + */
> + unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr,
> + DUMMY_BUF_SIZE, METADATA_PAD);
> + }
> +
> + /* Clear all queued buffers for the node */
> + unicam_return_buffers(node);
> +}
> +
> +static int unicam_enum_input(struct file *file, void *priv,
> + struct v4l2_input *inp)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + if (inp->index != 0)
> + return -EINVAL;
> +
> + inp->type = V4L2_INPUT_TYPE_CAMERA;
> + if (v4l2_subdev_has_op(dev->sensor, video, s_dv_timings)) {
> + inp->capabilities = V4L2_IN_CAP_DV_TIMINGS;
> + inp->std = 0;
> + } else if (v4l2_subdev_has_op(dev->sensor, video, s_std)) {
> + inp->capabilities = V4L2_IN_CAP_STD;
> + if (v4l2_subdev_call(dev->sensor, video, g_tvnorms, &inp->std)
> + < 0)
> + inp->std = V4L2_STD_ALL;
> + } else {
> + inp->capabilities = 0;
> + inp->std = 0;
> + }
> + sprintf(inp->name, "Camera 0");
strscpy()?
> + return 0;
> +}
> +
> +static int unicam_g_input(struct file *file, void *priv, unsigned int *i)
> +{
> + *i = 0;
> +
> + return 0;
> +}
> +
> +static int unicam_s_input(struct file *file, void *priv, unsigned int i)
> +{
> + /*
> + * FIXME: Ideally we would like to be able to query the source
> + * subdevice for information over the input connectors it supports,
> + * and map that through in to a call to video_ops->s_routing.
> + * There is no infrastructure support for defining that within
> + * devicetree at present. Until that is implemented we can't
> + * map a user physical connector number to s_routing input number.
> + */
> + if (i > 0)
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int unicam_querystd(struct file *file, void *priv,
> + v4l2_std_id *std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, querystd, std);
> +}
> +
> +static int unicam_g_std(struct file *file, void *priv, v4l2_std_id *std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, g_std, std);
> +}
> +
> +static int unicam_s_std(struct file *file, void *priv, v4l2_std_id std)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + int ret;
> + v4l2_std_id current_std;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, g_std, ¤t_std);
> + if (ret)
> + return ret;
> +
> + if (std == current_std)
> + return 0;
> +
> + if (vb2_is_busy(&node->buffer_queue))
> + return -EBUSY;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_std, std);
> +
> + /* Force recomputation of bytesperline */
> + node->v_fmt.fmt.pix.bytesperline = 0;
> +
> + unicam_reset_format(node);
> +
> + return ret;
> +}
> +
> +static int unicam_s_edid(struct file *file, void *priv, struct v4l2_edid *edid)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, set_edid, edid);
> +}
> +
> +static int unicam_g_edid(struct file *file, void *priv, struct v4l2_edid *edid)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, get_edid, edid);
> +}
> +
> +static int unicam_s_selection(struct file *file, void *priv,
> + struct v4l2_selection *sel)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_selection sdsel = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .target = sel->target,
> + .flags = sel->flags,
> + .r = sel->r,
> + };
> +
> + return v4l2_subdev_call(dev->sensor, pad, set_selection, NULL, &sdsel);
> +}
> +
> +static int unicam_g_selection(struct file *file, void *priv,
> + struct v4l2_selection *sel)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev_selection sdsel = {
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + .target = sel->target,
> + };
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, get_selection, NULL, &sdsel);
> + if (!ret)
> + sel->r = sdsel.r;
> +
> + return ret;
> +}
> +
> +static int unicam_enum_framesizes(struct file *file, void *priv,
> + struct v4l2_frmsizeenum *fsize)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + struct v4l2_subdev_frame_size_enum fse;
> + int ret;
> +
> + /* check for valid format */
> + fmt = find_format_by_pix(dev, fsize->pixel_format);
> + if (!fmt) {
> + unicam_dbg(3, dev, "Invalid pixel code: %x\n",
> + fsize->pixel_format);
> + return -EINVAL;
> + }
> + fse.code = fmt->code;
> +
> + fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
> + fse.index = fsize->index;
> + fse.pad = node->pad_id;
> +
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_size, NULL, &fse);
> + if (ret)
> + return ret;
> +
> + unicam_dbg(1, dev, "%s: index: %d code: %x W:[%d,%d] H:[%d,%d]\n",
> + __func__, fse.index, fse.code, fse.min_width, fse.max_width,
> + fse.min_height, fse.max_height);
> +
> + fsize->type = V4L2_FRMSIZE_TYPE_DISCRETE;
> + fsize->discrete.width = fse.max_width;
> + fsize->discrete.height = fse.max_height;
> +
> + return 0;
> +}
> +
> +static int unicam_enum_frameintervals(struct file *file, void *priv,
> + struct v4l2_frmivalenum *fival)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + const struct unicam_fmt *fmt;
> + struct v4l2_subdev_frame_interval_enum fie = {
> + .index = fival->index,
> + .width = fival->width,
> + .height = fival->height,
> + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> + };
> + int ret;
> +
> + fmt = find_format_by_pix(dev, fival->pixel_format);
> + if (!fmt)
> + return -EINVAL;
> +
> + fie.code = fmt->code;
> + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_interval,
> + NULL, &fie);
You're adding a new CSI-2 receiver driver but your driver appears to be
video node centric and does not use MC / V4L2 subdev uAPIs for pipeline
configuration.
This is effectively needed if you want to be able to capture embedded data.
I'd also recommend it since this way the driver will be compliant with all
camera sensor drivers, not just those that expose a single sub-device.
There are no good ways to change this once your driver is in upstream
kernel.
This is also why e.g. ipu3-cio2 driver is MC-centric.
> + if (ret)
> + return ret;
> +
> + fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
> + fival->discrete = fie.interval;
> +
> + return 0;
> +}
> +
> +static int unicam_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_g_parm_cap(video_devdata(file), dev->sensor, a);
> +}
> +
> +static int unicam_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_s_parm_cap(video_devdata(file), dev->sensor, a);
> +}
> +
> +static int unicam_g_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, g_dv_timings, timings);
> +}
> +
> +static int unicam_s_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_dv_timings current_timings;
> + int ret;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, g_dv_timings,
> + ¤t_timings);
> +
> + if (v4l2_match_dv_timings(timings, ¤t_timings, 0, false))
> + return 0;
> +
> + if (vb2_is_busy(&node->buffer_queue))
> + return -EBUSY;
> +
> + ret = v4l2_subdev_call(dev->sensor, video, s_dv_timings, timings);
> +
> + /* Force recomputation of bytesperline */
> + node->v_fmt.fmt.pix.bytesperline = 0;
> +
> + unicam_reset_format(node);
> +
> + return ret;
> +}
> +
> +static int unicam_query_dv_timings(struct file *file, void *priv,
> + struct v4l2_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, video, query_dv_timings, timings);
> +}
> +
> +static int unicam_enum_dv_timings(struct file *file, void *priv,
> + struct v4l2_enum_dv_timings *timings)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, enum_dv_timings, timings);
> +}
> +
> +static int unicam_dv_timings_cap(struct file *file, void *priv,
> + struct v4l2_dv_timings_cap *cap)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> +
> + return v4l2_subdev_call(dev->sensor, pad, dv_timings_cap, cap);
> +}
> +
> +static int unicam_subscribe_event(struct v4l2_fh *fh,
> + const struct v4l2_event_subscription *sub)
> +{
> + switch (sub->type) {
> + case V4L2_EVENT_FRAME_SYNC:
> + return v4l2_event_subscribe(fh, sub, 2, NULL);
> + case V4L2_EVENT_SOURCE_CHANGE:
> + return v4l2_event_subscribe(fh, sub, 4, NULL);
> + }
> +
> + return v4l2_ctrl_subscribe_event(fh, sub);
> +}
> +
> +static int unicam_log_status(struct file *file, void *fh)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + u32 reg;
> +
> + /* status for sub devices */
> + v4l2_device_call_all(&dev->v4l2_dev, 0, core, log_status);
> +
> + unicam_info(dev, "-----Receiver status-----\n");
> + unicam_info(dev, "V4L2 width/height: %ux%u\n",
> + node->v_fmt.fmt.pix.width, node->v_fmt.fmt.pix.height);
> + unicam_info(dev, "Mediabus format: %08x\n", node->fmt->code);
> + unicam_info(dev, "V4L2 format: %08x\n",
> + node->v_fmt.fmt.pix.pixelformat);
> + reg = reg_read(dev, UNICAM_IPIPE);
> + unicam_info(dev, "Unpacking/packing: %u / %u\n",
> + get_field(reg, UNICAM_PUM_MASK),
> + get_field(reg, UNICAM_PPM_MASK));
> + unicam_info(dev, "----Live data----\n");
> + unicam_info(dev, "Programmed stride: %4u\n",
> + reg_read(dev, UNICAM_IBLS));
> + unicam_info(dev, "Detected resolution: %ux%u\n",
> + reg_read(dev, UNICAM_IHSTA),
> + reg_read(dev, UNICAM_IVSTA));
> + unicam_info(dev, "Write pointer: %08x\n",
> + reg_read(dev, UNICAM_IBWP));
> +
> + return 0;
> +}
> +
> +static void unicam_notify(struct v4l2_subdev *sd,
> + unsigned int notification, void *arg)
> +{
> + struct unicam_device *dev = to_unicam_device(sd->v4l2_dev);
> +
> + switch (notification) {
> + case V4L2_DEVICE_NOTIFY_EVENT:
> + v4l2_event_queue(&dev->node[IMAGE_PAD].video_dev, arg);
> + break;
> + default:
> + break;
> + }
> +}
> +
> +static const struct vb2_ops unicam_video_qops = {
> + .wait_prepare = vb2_ops_wait_prepare,
> + .wait_finish = vb2_ops_wait_finish,
> + .queue_setup = unicam_queue_setup,
> + .buf_prepare = unicam_buffer_prepare,
> + .buf_queue = unicam_buffer_queue,
> + .start_streaming = unicam_start_streaming,
> + .stop_streaming = unicam_stop_streaming,
> +};
> +
> +/*
> + * unicam_v4l2_open : This function is based on the v4l2_fh_open helper
> + * function. It has been augmented to handle sensor subdevice power management,
> + */
> +static int unicam_v4l2_open(struct file *file)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + int ret;
> +
> + mutex_lock(&node->lock);
> +
> + ret = v4l2_fh_open(file);
> + if (ret) {
> + unicam_err(dev, "v4l2_fh_open failed\n");
> + goto unlock;
> + }
> +
> + node->open++;
> +
> + if (!v4l2_fh_is_singular_file(file))
> + goto unlock;
> +
> + ret = v4l2_subdev_call(dev->sensor, core, s_power, 1);
> + if (ret < 0 && ret != -ENOIOCTLCMD) {
> + v4l2_fh_release(file);
> + node->open--;
> + goto unlock;
> + }
> +
> + ret = 0;
> +
> +unlock:
> + mutex_unlock(&node->lock);
> + return ret;
> +}
> +
> +static int unicam_v4l2_release(struct file *file)
> +{
> + struct unicam_node *node = video_drvdata(file);
> + struct unicam_device *dev = node->dev;
> + struct v4l2_subdev *sd = dev->sensor;
> + bool fh_singular;
> + int ret;
> +
> + mutex_lock(&node->lock);
> +
> + fh_singular = v4l2_fh_is_singular_file(file);
> +
> + ret = _vb2_fop_release(file, NULL);
> +
> + if (fh_singular)
> + v4l2_subdev_call(sd, core, s_power, 0);
> +
> + node->open--;
> + mutex_unlock(&node->lock);
> +
> + return ret;
> +}
> +
> +/* unicam capture driver file operations */
> +static const struct v4l2_file_operations unicam_fops = {
> + .owner = THIS_MODULE,
> + .open = unicam_v4l2_open,
> + .release = unicam_v4l2_release,
> + .read = vb2_fop_read,
> + .poll = vb2_fop_poll,
> + .unlocked_ioctl = video_ioctl2,
> + .mmap = vb2_fop_mmap,
> +};
> +
> +/* unicam capture ioctl operations */
> +static const struct v4l2_ioctl_ops unicam_ioctl_ops = {
> + .vidioc_querycap = unicam_querycap,
> + .vidioc_enum_fmt_vid_cap = unicam_enum_fmt_vid_cap,
> + .vidioc_g_fmt_vid_cap = unicam_g_fmt_vid_cap,
> + .vidioc_s_fmt_vid_cap = unicam_s_fmt_vid_cap,
> + .vidioc_try_fmt_vid_cap = unicam_try_fmt_vid_cap,
> +
> + .vidioc_enum_fmt_meta_cap = unicam_enum_fmt_meta_cap,
> + .vidioc_g_fmt_meta_cap = unicam_g_fmt_meta_cap,
> + .vidioc_s_fmt_meta_cap = unicam_g_fmt_meta_cap,
> + .vidioc_try_fmt_meta_cap = unicam_g_fmt_meta_cap,
> +
> + .vidioc_enum_input = unicam_enum_input,
> + .vidioc_g_input = unicam_g_input,
> + .vidioc_s_input = unicam_s_input,
> +
> + .vidioc_querystd = unicam_querystd,
> + .vidioc_s_std = unicam_s_std,
> + .vidioc_g_std = unicam_g_std,
> +
> + .vidioc_g_edid = unicam_g_edid,
> + .vidioc_s_edid = unicam_s_edid,
> +
> + .vidioc_enum_framesizes = unicam_enum_framesizes,
> + .vidioc_enum_frameintervals = unicam_enum_frameintervals,
> +
> + .vidioc_g_selection = unicam_g_selection,
> + .vidioc_s_selection = unicam_s_selection,
> +
> + .vidioc_g_parm = unicam_g_parm,
> + .vidioc_s_parm = unicam_s_parm,
> +
> + .vidioc_s_dv_timings = unicam_s_dv_timings,
> + .vidioc_g_dv_timings = unicam_g_dv_timings,
> + .vidioc_query_dv_timings = unicam_query_dv_timings,
> + .vidioc_enum_dv_timings = unicam_enum_dv_timings,
> + .vidioc_dv_timings_cap = unicam_dv_timings_cap,
> +
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +
> + .vidioc_log_status = unicam_log_status,
> + .vidioc_subscribe_event = unicam_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +static int
> +unicam_async_bound(struct v4l2_async_notifier *notifier,
> + struct v4l2_subdev *subdev,
> + struct v4l2_async_subdev *asd)
> +{
> + struct unicam_device *unicam = to_unicam_device(notifier->v4l2_dev);
> +
> + if (unicam->sensor) {
> + unicam_info(unicam, "Rejecting subdev %s (Already set!!)",
> + subdev->name);
> + return 0;
> + }
> +
> + unicam->sensor = subdev;
> + unicam_dbg(1, unicam, "Using sensor %s for capture\n", subdev->name);
> +
> + return 0;
> +}
> +
> +static void unicam_release(struct kref *kref)
> +{
> + struct unicam_device *unicam =
> + container_of(kref, struct unicam_device, kref);
> +
> + v4l2_ctrl_handler_free(&unicam->ctrl_handler);
> + media_device_cleanup(&unicam->mdev);
> +
> + if (unicam->sensor_config)
> + v4l2_subdev_free_pad_config(unicam->sensor_config);
> +
> + kfree(unicam);
> +}
> +
> +static void unicam_put(struct unicam_device *unicam)
> +{
> + kref_put(&unicam->kref, unicam_release);
> +}
> +
> +static void unicam_get(struct unicam_device *unicam)
> +{
> + kref_get(&unicam->kref);
> +}
> +
> +static void unicam_node_release(struct video_device *vdev)
> +{
> + struct unicam_node *node = video_get_drvdata(vdev);
> +
> + unicam_put(node->dev);
> +}
> +
> +static int register_node(struct unicam_device *unicam, struct unicam_node *node,
> + enum v4l2_buf_type type, int pad_id)
> +{
> + struct video_device *vdev;
> + struct vb2_queue *q;
> + struct v4l2_mbus_framefmt mbus_fmt = {0};
> + const struct unicam_fmt *fmt;
> + int ret;
> +
> + if (pad_id == IMAGE_PAD) {
> + ret = __subdev_get_format(unicam, &mbus_fmt, pad_id);
> + if (ret) {
> + unicam_err(unicam, "Failed to get_format - ret %d\n",
> + ret);
> + return ret;
> + }
> +
> + fmt = find_format_by_code(mbus_fmt.code);
> + if (!fmt) {
> + /*
> + * Find the first format that the sensor and unicam both
> + * support
> + */
> + fmt = get_first_supported_format(unicam);
> +
> + if (!fmt)
> + /* No compatible formats */
> + return -EINVAL;
> +
> + mbus_fmt.code = fmt->code;
> + ret = __subdev_set_format(unicam, &mbus_fmt, pad_id);
> + if (ret)
> + return -EINVAL;
> + }
> + if (mbus_fmt.field != V4L2_FIELD_NONE) {
> + /* Interlaced not supported - disable it now. */
> + mbus_fmt.field = V4L2_FIELD_NONE;
> + ret = __subdev_set_format(unicam, &mbus_fmt, pad_id);
> + if (ret)
> + return -EINVAL;
> + }
> +
> + node->v_fmt.fmt.pix.pixelformat = fmt->fourcc ? fmt->fourcc
> + : fmt->repacked_fourcc;
> + } else {
> + /* Fix this node format as embedded data. */
> + fmt = find_format_by_code(MEDIA_BUS_FMT_SENSOR_DATA);
> + node->v_fmt.fmt.meta.dataformat = fmt->fourcc;
> + }
> +
> + node->dev = unicam;
> + node->pad_id = pad_id;
> + node->fmt = fmt;
> +
> + /* Read current subdev format */
> + unicam_reset_format(node);
> +
> + if (v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
> + v4l2_std_id tvnorms;
> +
> + if (WARN_ON(!v4l2_subdev_has_op(unicam->sensor, video,
> + g_tvnorms)))
> + /*
> + * Subdevice should not advertise s_std but not
> + * g_tvnorms
> + */
> + return -EINVAL;
> +
> + ret = v4l2_subdev_call(unicam->sensor, video,
> + g_tvnorms, &tvnorms);
> + if (WARN_ON(ret))
> + return -EINVAL;
> + node->video_dev.tvnorms |= tvnorms;
> + }
> +
> + spin_lock_init(&node->dma_queue_lock);
> + mutex_init(&node->lock);
> +
> + vdev = &node->video_dev;
> + if (pad_id == IMAGE_PAD) {
> + /* Add controls from the subdevice */
> + ret = v4l2_ctrl_add_handler(&unicam->ctrl_handler,
> + unicam->sensor->ctrl_handler, NULL,
> + true);
> + if (ret < 0)
> + return ret;
> +
> + /*
> + * If the sensor subdevice has any controls, associate the node
> + * with the ctrl handler to allow access from userland.
> + */
> + if (!list_empty(&unicam->ctrl_handler.ctrls))
> + vdev->ctrl_handler = &unicam->ctrl_handler;
> + }
> +
> + q = &node->buffer_queue;
> + q->type = type;
> + q->io_modes = VB2_MMAP | VB2_DMABUF | VB2_READ;
> + q->drv_priv = node;
> + q->ops = &unicam_video_qops;
> + q->mem_ops = &vb2_dma_contig_memops;
> + q->buf_struct_size = sizeof(struct unicam_buffer);
> + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
> + q->lock = &node->lock;
> + q->min_buffers_needed = 2;
> + q->dev = &unicam->pdev->dev;
> +
> + ret = vb2_queue_init(q);
> + if (ret) {
> + unicam_err(unicam, "vb2_queue_init() failed\n");
> + return ret;
> + }
> +
> + INIT_LIST_HEAD(&node->dma_queue);
> +
> + vdev->release = unicam_node_release;
> + vdev->fops = &unicam_fops;
> + vdev->ioctl_ops = &unicam_ioctl_ops;
> + vdev->v4l2_dev = &unicam->v4l2_dev;
> + vdev->vfl_dir = VFL_DIR_RX;
> + vdev->queue = q;
> + vdev->lock = &node->lock;
> + vdev->device_caps = (pad_id == IMAGE_PAD) ?
> + (V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING) :
> + (V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING);
> +
> + /* Define the device names */
> + snprintf(vdev->name, sizeof(vdev->name), "%s-%s", UNICAM_MODULE_NAME,
> + pad_id == IMAGE_PAD ? "image" : "embedded");
> +
> + video_set_drvdata(vdev, node);
> + if (pad_id == IMAGE_PAD)
> + vdev->entity.flags |= MEDIA_ENT_FL_DEFAULT;
> + node->pad.flags = MEDIA_PAD_FL_SINK;
> + media_entity_pads_init(&vdev->entity, 1, &node->pad);
> +
> + node->dummy_buf_cpu_addr = dma_alloc_coherent(&unicam->pdev->dev,
> + DUMMY_BUF_SIZE,
> + &node->dummy_buf_dma_addr,
> + GFP_KERNEL);
> + if (!node->dummy_buf_cpu_addr) {
> + unicam_err(unicam, "Unable to allocate dummy buffer.\n");
> + return -ENOMEM;
> + }
> +
> + if (pad_id == METADATA_PAD) {
> + v4l2_disable_ioctl(vdev, VIDIOC_DQEVENT);
> + v4l2_disable_ioctl(vdev, VIDIOC_SUBSCRIBE_EVENT);
> + v4l2_disable_ioctl(vdev, VIDIOC_UNSUBSCRIBE_EVENT);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_std)) {
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_STD);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_STD);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUMSTD);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, querystd))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_QUERYSTD);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_dv_timings)) {
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_EDID);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_EDID);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_DV_TIMINGS_CAP);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUM_DV_TIMINGS);
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_QUERY_DV_TIMINGS);
> + }
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, enum_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev,
> + VIDIOC_ENUM_FRAMEINTERVALS);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, g_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_PARM);
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, video, s_frame_interval))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_PARM);
> +
> + if (pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, enum_frame_size))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_ENUM_FRAMESIZES);
> +
> + if (node->pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, set_selection))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_S_SELECTION);
> +
> + if (node->pad_id == METADATA_PAD ||
> + !v4l2_subdev_has_op(unicam->sensor, pad, get_selection))
> + v4l2_disable_ioctl(&node->video_dev, VIDIOC_G_SELECTION);
> +
> + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1);
> + if (ret) {
> + unicam_err(unicam, "Unable to register video device %s\n",
> + vdev->name);
> + return ret;
> + }
> +
> + /*
> + * Acquire a reference to unicam, which will be released when the video
> + * device will be unregistered and userspace will have closed all open
> + * file handles.
> + */
> + unicam_get(unicam);
> + node->registered = true;
> +
> + if (pad_id != METADATA_PAD || unicam->sensor_embedded_data) {
> + ret = media_create_pad_link(&unicam->sensor->entity, pad_id,
> + &node->video_dev.entity, 0,
> + MEDIA_LNK_FL_ENABLED |
> + MEDIA_LNK_FL_IMMUTABLE);
This does create two links between the sensor and the CSI-2 receiver,
doesn't it?
The links in Media controller represent physical links, not logical flows
of data. At the time the API was added, it wasn't thought there would be a
need to separate the two.
There is an effort to add the concept of data flow to V4L2, but it's been
complicated and we haven't had patches for the CSI-2 receiver driver to
support it. Perhaps Unicam could be the first one to do that?
Alternatively support for embedded data could be removed in the meantime.
The latest patchset is here I believe:
<URL:https://patchwork.kernel.org/project/linux-media/list/?series=98277>
> + if (ret)
> + unicam_err(unicam, "Unable to create pad link for %s\n",
> + vdev->name);
> + }
> +
> + return ret;
> +}
> +
> +static void unregister_nodes(struct unicam_device *unicam)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> + struct unicam_node *node = &unicam->node[i];
> +
> + if (node->dummy_buf_cpu_addr) {
> + dma_free_coherent(&unicam->pdev->dev, DUMMY_BUF_SIZE,
> + node->dummy_buf_cpu_addr,
> + node->dummy_buf_dma_addr);
> + }
> +
> + if (node->registered) {
> + node->registered = false;
> + video_unregister_device(&node->video_dev);
> + }
> + }
> +}
> +
> +static int unicam_probe_complete(struct unicam_device *unicam)
> +{
> + int ret;
> +
> + unicam->v4l2_dev.notify = unicam_notify;
> +
> + unicam->sensor_config = v4l2_subdev_alloc_pad_config(unicam->sensor);
> + if (!unicam->sensor_config)
> + return -ENOMEM;
> +
> + unicam->sensor_embedded_data = (unicam->sensor->entity.num_pads >= 2);
> +
> + ret = register_node(unicam, &unicam->node[IMAGE_PAD],
> + V4L2_BUF_TYPE_VIDEO_CAPTURE, IMAGE_PAD);
> + if (ret) {
> + unicam_err(unicam, "Unable to register image video device.\n");
> + goto unregister;
> + }
> +
> + ret = register_node(unicam, &unicam->node[METADATA_PAD],
> + V4L2_BUF_TYPE_META_CAPTURE, METADATA_PAD);
> + if (ret) {
> + unicam_err(unicam, "Unable to register metadata video device.\n");
> + goto unregister;
> + }
> +
> + ret = v4l2_device_register_ro_subdev_nodes(&unicam->v4l2_dev);
> + if (ret) {
> + unicam_err(unicam, "Unable to register subdev nodes.\n");
> + goto unregister;
> + }
> +
> + /*
> + * Release the initial reference, all references are now owned by the
> + * video devices.
> + */
> + unicam_put(unicam);
> + return 0;
> +
> +unregister:
> + unregister_nodes(unicam);
> + unicam_put(unicam);
> +
> + return ret;
> +}
> +
> +static int unicam_async_complete(struct v4l2_async_notifier *notifier)
> +{
> + struct unicam_device *unicam = to_unicam_device(notifier->v4l2_dev);
> +
> + return unicam_probe_complete(unicam);
No need for such a wrapper.
> +}
> +
> +static const struct v4l2_async_notifier_operations unicam_async_ops = {
> + .bound = unicam_async_bound,
> + .complete = unicam_async_complete,
> +};
> +
> +static int of_unicam_connect_subdevs(struct unicam_device *dev)
> +{
> + struct platform_device *pdev = dev->pdev;
> + struct v4l2_fwnode_endpoint ep = { 0 };
> + struct device_node *ep_node;
> + struct device_node *sensor_node;
> + unsigned int lane;
> + int ret = -EINVAL;
> +
> + if (of_property_read_u32(pdev->dev.of_node, "brcm,num-data-lanes",
> + &dev->max_data_lanes) < 0) {
> + unicam_err(dev, "number of data lanes not set\n");
> + return -EINVAL;
> + }
> +
> + /* Get the local endpoint and remote device. */
> + ep_node = of_graph_get_next_endpoint(pdev->dev.of_node, NULL);
> + if (!ep_node) {
> + unicam_dbg(3, dev, "can't get next endpoint\n");
> + return -EINVAL;
> + }
> +
> + unicam_dbg(3, dev, "ep_node is %pOF\n", ep_node);
v4l2_fwnode_endpoint_parse() already prints this.
> +
> + sensor_node = of_graph_get_remote_port_parent(ep_node);
> + if (!sensor_node) {
> + unicam_dbg(3, dev, "can't get remote parent\n");
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(1, dev, "found subdevice %pOF\n", sensor_node);
> +
> + /* Parse the local endpoint and validate its configuration. */
> + v4l2_fwnode_endpoint_parse(of_fwnode_handle(ep_node), &ep);
> +
> + unicam_dbg(3, dev, "parsed local endpoint, bus_type %u\n",
> + ep.bus_type);
> +
> + dev->bus_type = ep.bus_type;
> +
> + switch (ep.bus_type) {
> + case V4L2_MBUS_CSI2_DPHY:
> + switch (ep.bus.mipi_csi2.num_data_lanes) {
> + case 1:
> + case 2:
> + case 4:
> + break;
> +
> + default:
> + unicam_err(dev, "subdevice %pOF: %u data lanes not supported\n",
> + sensor_node,
> + ep.bus.mipi_csi2.num_data_lanes);
> + goto cleanup_exit;
> + }
> +
> + for (lane = 0; lane < ep.bus.mipi_csi2.num_data_lanes; lane++) {
> + if (ep.bus.mipi_csi2.data_lanes[lane] != lane + 1) {
> + unicam_err(dev, "subdevice %pOF: data lanes reordering not supported\n",
> + sensor_node);
I'd just omit the check. It's up to the board designer really. No other
drivers check for this either (apart from those supporting lane remapping).
> + goto cleanup_exit;
> + }
> + }
> +
> + if (ep.bus.mipi_csi2.num_data_lanes > dev->max_data_lanes) {
> + unicam_err(dev, "subdevice requires %u data lanes when %u are supported\n",
> + ep.bus.mipi_csi2.num_data_lanes,
> + dev->max_data_lanes);
> + }
> +
> + dev->max_data_lanes = ep.bus.mipi_csi2.num_data_lanes;
> + dev->bus_flags = ep.bus.mipi_csi2.flags;
> +
> + break;
> +
> + case V4L2_MBUS_CCP2:
> + if (ep.bus.mipi_csi1.clock_lane != 0 ||
> + ep.bus.mipi_csi1.data_lane != 1) {
> + unicam_err(dev, "subdevice %pOF: unsupported lanes configuration\n",
> + sensor_node);
> + goto cleanup_exit;
> + }
> +
> + dev->max_data_lanes = 1;
> + dev->bus_flags = ep.bus.mipi_csi1.strobe;
> + break;
> +
> + default:
> + /* Unsupported bus type */
> + unicam_err(dev, "subdevice %pOF: unsupported bus type %u\n",
> + sensor_node, ep.bus_type);
> + goto cleanup_exit;
> + }
> +
> + unicam_dbg(3, dev, "subdevice %pOF: %s bus, %u data lanes, flags=0x%08x\n",
> + sensor_node,
> + dev->bus_type == V4L2_MBUS_CSI2_DPHY ? "CSI-2" : "CCP2",
> + dev->max_data_lanes, dev->bus_flags);
> +
> + /* Initialize and register the async notifier. */
> + v4l2_async_notifier_init(&dev->notifier);
> + dev->notifier.ops = &unicam_async_ops;
> +
> + dev->asd.match_type = V4L2_ASYNC_MATCH_FWNODE;
> + dev->asd.match.fwnode = of_fwnode_handle(sensor_node);
> + ret = v4l2_async_notifier_add_subdev(&dev->notifier, &dev->asd);
You could use v4l2_async_notifier_add_fwnode_subdev().
Do note that when the async subdev is released, the containing struct will
be, too. And asd needs to be the first field there.
> + if (ret) {
> + unicam_err(dev, "Error adding subdevice: %d\n", ret);
> + goto cleanup_exit;
> + }
> +
> + ret = v4l2_async_notifier_register(&dev->v4l2_dev, &dev->notifier);
> + if (ret) {
> + unicam_err(dev, "Error registering async notifier: %d\n", ret);
> + ret = -EINVAL;
> + }
> +
> +cleanup_exit:
> + of_node_put(sensor_node);
> + of_node_put(ep_node);
> +
> + return ret;
> +}
> +
> +static int unicam_probe(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam;
> + int ret;
> +
> + unicam = kzalloc(sizeof(*unicam), GFP_KERNEL);
devm_kzalloc() would not hurt...
> + if (!unicam)
> + return -ENOMEM;
> +
> + kref_init(&unicam->kref);
> + unicam->pdev = pdev;
> +
> + unicam->base = devm_platform_ioremap_resource(pdev, 0);
...as you're already using devm_*() functions.
> + if (IS_ERR(unicam->base)) {
> + unicam_err(unicam, "Failed to get main io block\n");
> + ret = PTR_ERR(unicam->base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clk_gate_base = devm_platform_ioremap_resource(pdev, 1);
> + if (IS_ERR(unicam->clk_gate_base)) {
> + unicam_err(unicam, "Failed to get 2nd io block\n");
> + ret = PTR_ERR(unicam->clk_gate_base);
> + goto err_unicam_put;
> + }
> +
> + unicam->clock = devm_clk_get(&pdev->dev, "lp");
> + if (IS_ERR(unicam->clock)) {
> + unicam_err(unicam, "Failed to get clock\n");
> + ret = PTR_ERR(unicam->clock);
> + goto err_unicam_put;
> + }
> +
> + ret = platform_get_irq(pdev, 0);
> + if (ret <= 0) {
> + dev_err(&pdev->dev, "No IRQ resource\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + ret = devm_request_irq(&pdev->dev, ret, unicam_isr, 0,
> + "unicam_capture0", unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Unable to request interrupt\n");
> + ret = -EINVAL;
> + goto err_unicam_put;
> + }
> +
> + unicam->mdev.dev = &pdev->dev;
> + strscpy(unicam->mdev.model, UNICAM_MODULE_NAME,
I think this should be about hardware, not the driver.
> + sizeof(unicam->mdev.model));
> + strscpy(unicam->mdev.serial, "", sizeof(unicam->mdev.serial));
> + snprintf(unicam->mdev.bus_info, sizeof(unicam->mdev.bus_info),
> + "platform:%s", dev_name(&pdev->dev));
> + unicam->mdev.hw_revision = 0;
> +
> + media_device_init(&unicam->mdev);
> +
> + unicam->v4l2_dev.mdev = &unicam->mdev;
> +
> + ret = v4l2_device_register(&pdev->dev, &unicam->v4l2_dev);
> + if (ret) {
> + unicam_err(unicam,
> + "Unable to register v4l2 device.\n");
> + goto err_unicam_put;
> + }
> +
> + ret = media_device_register(&unicam->mdev);
> + if (ret < 0) {
> + unicam_err(unicam,
> + "Unable to register media-controller device.\n");
> + goto err_v4l2_unregister;
> + }
> +
> + /* Reserve space for the controls */
> + ret = v4l2_ctrl_handler_init(&unicam->ctrl_handler, 16);
> + if (ret < 0)
> + goto err_media_unregister;
> +
> + /* set the driver data in platform device */
> + platform_set_drvdata(pdev, unicam);
> +
> + ret = of_unicam_connect_subdevs(unicam);
> + if (ret) {
> + dev_err(&pdev->dev, "Failed to connect subdevs\n");
> + goto err_media_unregister;
> + }
> +
> + /* Enable the block power domain */
Hmm.
> + pm_runtime_enable(&pdev->dev);
> +
> + return 0;
> +
> +err_media_unregister:
> + media_device_unregister(&unicam->mdev);
> +err_v4l2_unregister:
> + v4l2_device_unregister(&unicam->v4l2_dev);
> +err_unicam_put:
> + unicam_put(unicam);
> +
> + return ret;
> +}
> +
> +static int unicam_remove(struct platform_device *pdev)
> +{
> + struct unicam_device *unicam = platform_get_drvdata(pdev);
> +
> + unicam_dbg(2, unicam, "%s\n", __func__);
This looks like development time leftovers.
> +
> + v4l2_async_notifier_unregister(&unicam->notifier);
> + v4l2_device_unregister(&unicam->v4l2_dev);
> + media_device_unregister(&unicam->mdev);
> + unregister_nodes(unicam);
> +
> + pm_runtime_disable(&pdev->dev);
> +
> + return 0;
> +}
> +
> +static const struct of_device_id unicam_of_match[] = {
> + { .compatible = "brcm,bcm2835-unicam", },
> + { /* sentinel */ },
> +};
> +MODULE_DEVICE_TABLE(of, unicam_of_match);
> +
> +static struct platform_driver unicam_driver = {
> + .probe = unicam_probe,
> + .remove = unicam_remove,
> + .driver = {
> + .name = UNICAM_MODULE_NAME,
> + .of_match_table = of_match_ptr(unicam_of_match),
> + },
> +};
> +
> +module_platform_driver(unicam_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson <dave.stevenson@raspberrypi.com>");
> +MODULE_DESCRIPTION("BCM2835 Unicam driver");
> +MODULE_LICENSE("GPL");
> +MODULE_VERSION(UNICAM_VERSION);
> diff --git a/drivers/media/platform/bcm2835/vc4-regs-unicam.h b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> new file mode 100644
> index 000000000000..ae059a171d0f
> --- /dev/null
> +++ b/drivers/media/platform/bcm2835/vc4-regs-unicam.h
> @@ -0,0 +1,253 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +
> +/*
> + * Copyright (C) 2017-2020 Raspberry Pi Trading.
> + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> + */
> +
> +#ifndef VC4_REGS_UNICAM_H
> +#define VC4_REGS_UNICAM_H
> +
> +/*
> + * The following values are taken from files found within the code drop
> + * made by Broadcom for the BCM21553 Graphics Driver, predominantly in
> + * brcm_usrlib/dag/vmcsx/vcinclude/hardware_vc4.h.
> + * They have been modified to be only the register offset.
> + */
> +#define UNICAM_CTRL 0x000
> +#define UNICAM_STA 0x004
> +#define UNICAM_ANA 0x008
> +#define UNICAM_PRI 0x00c
> +#define UNICAM_CLK 0x010
> +#define UNICAM_CLT 0x014
> +#define UNICAM_DAT0 0x018
> +#define UNICAM_DAT1 0x01c
> +#define UNICAM_DAT2 0x020
> +#define UNICAM_DAT3 0x024
> +#define UNICAM_DLT 0x028
> +#define UNICAM_CMP0 0x02c
> +#define UNICAM_CMP1 0x030
> +#define UNICAM_CAP0 0x034
> +#define UNICAM_CAP1 0x038
> +#define UNICAM_ICTL 0x100
> +#define UNICAM_ISTA 0x104
> +#define UNICAM_IDI0 0x108
> +#define UNICAM_IPIPE 0x10c
> +#define UNICAM_IBSA0 0x110
> +#define UNICAM_IBEA0 0x114
> +#define UNICAM_IBLS 0x118
> +#define UNICAM_IBWP 0x11c
> +#define UNICAM_IHWIN 0x120
> +#define UNICAM_IHSTA 0x124
> +#define UNICAM_IVWIN 0x128
> +#define UNICAM_IVSTA 0x12c
> +#define UNICAM_ICC 0x130
> +#define UNICAM_ICS 0x134
> +#define UNICAM_IDC 0x138
> +#define UNICAM_IDPO 0x13c
> +#define UNICAM_IDCA 0x140
> +#define UNICAM_IDCD 0x144
> +#define UNICAM_IDS 0x148
> +#define UNICAM_DCS 0x200
> +#define UNICAM_DBSA0 0x204
> +#define UNICAM_DBEA0 0x208
> +#define UNICAM_DBWP 0x20c
> +#define UNICAM_DBCTL 0x300
> +#define UNICAM_IBSA1 0x304
> +#define UNICAM_IBEA1 0x308
> +#define UNICAM_IDI1 0x30c
> +#define UNICAM_DBSA1 0x310
> +#define UNICAM_DBEA1 0x314
> +#define UNICAM_MISC 0x400
> +
> +/*
> + * The following bitmasks are from the kernel released by Broadcom
> + * for Android - https://android.googlesource.com/kernel/bcm/
> + * The Rhea, Hawaii, and Java chips all contain the same VideoCore4
> + * Unicam block as BCM2835, as defined in eg
> + * arch/arm/mach-rhea/include/mach/rdb_A0/brcm_rdb_cam.h and similar.
> + * Values reworked to use the kernel BIT and GENMASK macros.
> + *
> + * Some of the bit mnenomics have been amended to match the datasheet.
> + */
> +/* UNICAM_CTRL Register */
> +#define UNICAM_CPE BIT(0)
> +#define UNICAM_MEM BIT(1)
> +#define UNICAM_CPR BIT(2)
> +#define UNICAM_CPM_MASK GENMASK(3, 3)
> +#define UNICAM_CPM_CSI2 0
> +#define UNICAM_CPM_CCP2 1
> +#define UNICAM_SOE BIT(4)
> +#define UNICAM_DCM_MASK GENMASK(5, 5)
> +#define UNICAM_DCM_STROBE 0
> +#define UNICAM_DCM_DATA 1
> +#define UNICAM_SLS BIT(6)
> +#define UNICAM_PFT_MASK GENMASK(11, 8)
> +#define UNICAM_OET_MASK GENMASK(20, 12)
> +
> +/* UNICAM_STA Register */
> +#define UNICAM_SYN BIT(0)
> +#define UNICAM_CS BIT(1)
> +#define UNICAM_SBE BIT(2)
> +#define UNICAM_PBE BIT(3)
> +#define UNICAM_HOE BIT(4)
> +#define UNICAM_PLE BIT(5)
> +#define UNICAM_SSC BIT(6)
> +#define UNICAM_CRCE BIT(7)
> +#define UNICAM_OES BIT(8)
> +#define UNICAM_IFO BIT(9)
> +#define UNICAM_OFO BIT(10)
> +#define UNICAM_BFO BIT(11)
> +#define UNICAM_DL BIT(12)
> +#define UNICAM_PS BIT(13)
> +#define UNICAM_IS BIT(14)
> +#define UNICAM_PI0 BIT(15)
> +#define UNICAM_PI1 BIT(16)
> +#define UNICAM_FSI_S BIT(17)
> +#define UNICAM_FEI_S BIT(18)
> +#define UNICAM_LCI_S BIT(19)
> +#define UNICAM_BUF0_RDY BIT(20)
> +#define UNICAM_BUF0_NO BIT(21)
> +#define UNICAM_BUF1_RDY BIT(22)
> +#define UNICAM_BUF1_NO BIT(23)
> +#define UNICAM_DI BIT(24)
> +
> +#define UNICAM_STA_MASK_ALL \
> + (UNICAM_DL + \
> + UNICAM_SBE + \
> + UNICAM_PBE + \
> + UNICAM_HOE + \
> + UNICAM_PLE + \
> + UNICAM_SSC + \
> + UNICAM_CRCE + \
> + UNICAM_IFO + \
> + UNICAM_OFO + \
> + UNICAM_PS + \
> + UNICAM_PI0 + \
> + UNICAM_PI1)
> +
> +/* UNICAM_ANA Register */
> +#define UNICAM_APD BIT(0)
> +#define UNICAM_BPD BIT(1)
> +#define UNICAM_AR BIT(2)
> +#define UNICAM_DDL BIT(3)
> +#define UNICAM_CTATADJ_MASK GENMASK(7, 4)
> +#define UNICAM_PTATADJ_MASK GENMASK(11, 8)
> +
> +/* UNICAM_PRI Register */
> +#define UNICAM_PE BIT(0)
> +#define UNICAM_PT_MASK GENMASK(2, 1)
> +#define UNICAM_NP_MASK GENMASK(7, 4)
> +#define UNICAM_PP_MASK GENMASK(11, 8)
> +#define UNICAM_BS_MASK GENMASK(15, 12)
> +#define UNICAM_BL_MASK GENMASK(17, 16)
> +
> +/* UNICAM_CLK Register */
> +#define UNICAM_CLE BIT(0)
> +#define UNICAM_CLPD BIT(1)
> +#define UNICAM_CLLPE BIT(2)
> +#define UNICAM_CLHSE BIT(3)
> +#define UNICAM_CLTRE BIT(4)
> +#define UNICAM_CLAC_MASK GENMASK(8, 5)
> +#define UNICAM_CLSTE BIT(29)
> +
> +/* UNICAM_CLT Register */
> +#define UNICAM_CLT1_MASK GENMASK(7, 0)
> +#define UNICAM_CLT2_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DATn Registers */
> +#define UNICAM_DLE BIT(0)
> +#define UNICAM_DLPD BIT(1)
> +#define UNICAM_DLLPE BIT(2)
> +#define UNICAM_DLHSE BIT(3)
> +#define UNICAM_DLTRE BIT(4)
> +#define UNICAM_DLSM BIT(5)
> +#define UNICAM_DLFO BIT(28)
> +#define UNICAM_DLSTE BIT(29)
> +
> +#define UNICAM_DAT_MASK_ALL (UNICAM_DLSTE + UNICAM_DLFO)
> +
> +/* UNICAM_DLT Register */
> +#define UNICAM_DLT1_MASK GENMASK(7, 0)
> +#define UNICAM_DLT2_MASK GENMASK(15, 8)
> +#define UNICAM_DLT3_MASK GENMASK(23, 16)
> +
> +/* UNICAM_ICTL Register */
> +#define UNICAM_FSIE BIT(0)
> +#define UNICAM_FEIE BIT(1)
> +#define UNICAM_IBOB BIT(2)
> +#define UNICAM_FCM BIT(3)
> +#define UNICAM_TFC BIT(4)
> +#define UNICAM_LIP_MASK GENMASK(6, 5)
> +#define UNICAM_LCIE_MASK GENMASK(28, 16)
> +
> +/* UNICAM_IDI0/1 Register */
> +#define UNICAM_ID0_MASK GENMASK(7, 0)
> +#define UNICAM_ID1_MASK GENMASK(15, 8)
> +#define UNICAM_ID2_MASK GENMASK(23, 16)
> +#define UNICAM_ID3_MASK GENMASK(31, 24)
> +
> +/* UNICAM_ISTA Register */
> +#define UNICAM_FSI BIT(0)
> +#define UNICAM_FEI BIT(1)
> +#define UNICAM_LCI BIT(2)
> +
> +#define UNICAM_ISTA_MASK_ALL (UNICAM_FSI + UNICAM_FEI + UNICAM_LCI)
> +
> +/* UNICAM_IPIPE Register */
> +#define UNICAM_PUM_MASK GENMASK(2, 0)
> + /* Unpacking modes */
> + #define UNICAM_PUM_NONE 0
> + #define UNICAM_PUM_UNPACK6 1
> + #define UNICAM_PUM_UNPACK7 2
> + #define UNICAM_PUM_UNPACK8 3
> + #define UNICAM_PUM_UNPACK10 4
> + #define UNICAM_PUM_UNPACK12 5
> + #define UNICAM_PUM_UNPACK14 6
> + #define UNICAM_PUM_UNPACK16 7
> +#define UNICAM_DDM_MASK GENMASK(6, 3)
> +#define UNICAM_PPM_MASK GENMASK(9, 7)
> + /* Packing modes */
> + #define UNICAM_PPM_NONE 0
> + #define UNICAM_PPM_PACK8 1
> + #define UNICAM_PPM_PACK10 2
> + #define UNICAM_PPM_PACK12 3
> + #define UNICAM_PPM_PACK14 4
> + #define UNICAM_PPM_PACK16 5
> +#define UNICAM_DEM_MASK GENMASK(11, 10)
> +#define UNICAM_DEBL_MASK GENMASK(14, 12)
> +#define UNICAM_ICM_MASK GENMASK(16, 15)
> +#define UNICAM_IDM_MASK GENMASK(17, 17)
> +
> +/* UNICAM_ICC Register */
> +#define UNICAM_ICFL_MASK GENMASK(4, 0)
> +#define UNICAM_ICFH_MASK GENMASK(9, 5)
> +#define UNICAM_ICST_MASK GENMASK(12, 10)
> +#define UNICAM_ICLT_MASK GENMASK(15, 13)
> +#define UNICAM_ICLL_MASK GENMASK(31, 16)
> +
> +/* UNICAM_DCS Register */
> +#define UNICAM_DIE BIT(0)
> +#define UNICAM_DIM BIT(1)
> +#define UNICAM_DBOB BIT(3)
> +#define UNICAM_FDE BIT(4)
> +#define UNICAM_LDP BIT(5)
> +#define UNICAM_EDL_MASK GENMASK(15, 8)
> +
> +/* UNICAM_DBCTL Register */
> +#define UNICAM_DBEN BIT(0)
> +#define UNICAM_BUF0_IE BIT(1)
> +#define UNICAM_BUF1_IE BIT(2)
> +
> +/* UNICAM_CMP[0,1] register */
> +#define UNICAM_PCE BIT(31)
> +#define UNICAM_GI BIT(9)
> +#define UNICAM_CPH BIT(8)
> +#define UNICAM_PCVC_MASK GENMASK(7, 6)
> +#define UNICAM_PCDT_MASK GENMASK(5, 0)
> +
> +/* UNICAM_MISC register */
> +#define UNICAM_FL0 BIT(6)
> +#define UNICAM_FL1 BIT(9)
> +
> +#endif
--
Kind regards,
Sakari Ailus
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-09-15 7:03 ` Sakari Ailus
@ 2020-09-15 9:32 ` Laurent Pinchart
2020-09-15 13:28 ` Dave Stevenson
2020-09-15 17:30 ` Dave Stevenson
1 sibling, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-09-15 9:32 UTC (permalink / raw)
To: Sakari Ailus
Cc: linux-media, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Sakari,
(With a question for Dave below)
I'm replying to the two main points of your review. All the other
comments look fine at a glance, Jacopo is having a more detailed look
and will incorporate them in v3.
On Tue, Sep 15, 2020 at 10:03:26AM +0300, Sakari Ailus wrote:
> Hi Laurent,
>
> Thanks for the patch, and my apologies for the late review. Please do cc me
> for v3.
>
> After a quick look I can already say this is the cleanest Unicam driver
> I've ever seen. But please also see my detailed comments below.
>
> On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > Compared to the bcm2835-camera driver present in staging, this driver
> > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > the VC4 firmware running on the VPU.
> >
> > The commit is made up of a series of changes cherry-picked from the
> > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > additional enhancements, forward-ported to the mainline kernel.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v1:
> >
> > - Re-fetch mbus code from subdev on a g_fmt call
> > - Group all ioctl disabling together
> > - Fix reference counting in unicam_open
> > - Add support for VIDIOC_[S|G]_SELECTION
> > ---
> > MAINTAINERS | 7 +
> > drivers/media/platform/Kconfig | 1 +
> > drivers/media/platform/Makefile | 2 +
> > drivers/media/platform/bcm2835/Kconfig | 15 +
> > drivers/media/platform/bcm2835/Makefile | 3 +
> > .../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
> > .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> > 7 files changed, 3106 insertions(+)
> > create mode 100644 drivers/media/platform/bcm2835/Kconfig
> > create mode 100644 drivers/media/platform/bcm2835/Makefile
> > create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> > create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
[snip]
> > diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > new file mode 100644
> > index 000000000000..2e9387cbc1e0
> > --- /dev/null
> > +++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > @@ -0,0 +1,2825 @@
[snip]
> > +static int unicam_enum_frameintervals(struct file *file, void *priv,
> > + struct v4l2_frmivalenum *fival)
> > +{
> > + struct unicam_node *node = video_drvdata(file);
> > + struct unicam_device *dev = node->dev;
> > + const struct unicam_fmt *fmt;
> > + struct v4l2_subdev_frame_interval_enum fie = {
> > + .index = fival->index,
> > + .width = fival->width,
> > + .height = fival->height,
> > + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > + };
> > + int ret;
> > +
> > + fmt = find_format_by_pix(dev, fival->pixel_format);
> > + if (!fmt)
> > + return -EINVAL;
> > +
> > + fie.code = fmt->code;
> > + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_interval,
> > + NULL, &fie);
>
> You're adding a new CSI-2 receiver driver but your driver appears to be
> video node centric and does not use MC / V4L2 subdev uAPIs for pipeline
> configuration.
>
> This is effectively needed if you want to be able to capture embedded data.
>
> I'd also recommend it since this way the driver will be compliant with all
> camera sensor drivers, not just those that expose a single sub-device.
> There are no good ways to change this once your driver is in upstream
> kernel.
>
> This is also why e.g. ipu3-cio2 driver is MC-centric.
I've had lengthy discussions with Dave on this topic. While I agree with
you in principle, Dave had good arguments for keeping this
video-node-centric. We all agreed it wasn't a perfect solution, but it
could still be a pragmatic one.
If I remember correctly the discussion was in private e-mails though.
Dave, I'm pretty sure you're tired of repeating the same thing, but
Sakari can't be expected to know all we've talked about. I can try to
summarize your points, but I may not do a very good job at defending
your point of view given that I wish you would be wrong :-) Would you
like to summarize your position, or should I give it a go ?
> > + if (ret)
> > + return ret;
> > +
> > + fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
> > + fival->discrete = fie.interval;
> > +
> > + return 0;
> > +}
[snip]
> > +static int register_node(struct unicam_device *unicam, struct unicam_node *node,
> > + enum v4l2_buf_type type, int pad_id)
> > +{
[snip]
> > + if (pad_id != METADATA_PAD || unicam->sensor_embedded_data) {
> > + ret = media_create_pad_link(&unicam->sensor->entity, pad_id,
> > + &node->video_dev.entity, 0,
> > + MEDIA_LNK_FL_ENABLED |
> > + MEDIA_LNK_FL_IMMUTABLE);
>
> This does create two links between the sensor and the CSI-2 receiver,
> doesn't it?
>
> The links in Media controller represent physical links, not logical flows
> of data. At the time the API was added, it wasn't thought there would be a
> need to separate the two.
>
> There is an effort to add the concept of data flow to V4L2, but it's been
> complicated and we haven't had patches for the CSI-2 receiver driver to
> support it. Perhaps Unicam could be the first one to do that?
I agree that this is the right approach. The V4L2 multiplexed streams
support seems to be one of these cursed series, where bad things happen
to anyone who touches it. I was about to actively start working on it
again back in June for a different project, which then got frozen at the
last minute :-S
Would you like to give it a try ? :-) I'd be more than happy to provide
you hardware as a present.
> Alternatively support for embedded data could be removed in the meantime.
>
> The latest patchset is here I believe:
>
> <URL:https://patchwork.kernel.org/project/linux-media/list/?series=98277>
>
> > + if (ret)
> > + unicam_err(unicam, "Unable to create pad link for %s\n",
> > + vdev->name);
> > + }
> > +
> > + return ret;
> > +}
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-09-15 9:32 ` Laurent Pinchart
@ 2020-09-15 13:28 ` Dave Stevenson
2020-10-30 17:53 ` Jacopo Mondi
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-09-15 13:28 UTC (permalink / raw)
To: Laurent Pinchart
Cc: Sakari Ailus, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
Hi Sakari & Laurent
On Tue, 15 Sep 2020 at 10:33, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Sakari,
>
> (With a question for Dave below)
>
> I'm replying to the two main points of your review. All the other
> comments look fine at a glance, Jacopo is having a more detailed look
> and will incorporate them in v3.
>
> On Tue, Sep 15, 2020 at 10:03:26AM +0300, Sakari Ailus wrote:
> > Hi Laurent,
> >
> > Thanks for the patch, and my apologies for the late review. Please do cc me
> > for v3.
> >
> > After a quick look I can already say this is the cleanest Unicam driver
> > I've ever seen. But please also see my detailed comments below.
> >
> > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > From: Naushir Patuck <naush@raspberrypi.com>
> > >
> > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > Compared to the bcm2835-camera driver present in staging, this driver
> > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > the VC4 firmware running on the VPU.
> > >
> > > The commit is made up of a series of changes cherry-picked from the
> > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > additional enhancements, forward-ported to the mainline kernel.
> > >
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > ---
> > > Changes since v1:
> > >
> > > - Re-fetch mbus code from subdev on a g_fmt call
> > > - Group all ioctl disabling together
> > > - Fix reference counting in unicam_open
> > > - Add support for VIDIOC_[S|G]_SELECTION
> > > ---
> > > MAINTAINERS | 7 +
> > > drivers/media/platform/Kconfig | 1 +
> > > drivers/media/platform/Makefile | 2 +
> > > drivers/media/platform/bcm2835/Kconfig | 15 +
> > > drivers/media/platform/bcm2835/Makefile | 3 +
> > > .../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
> > > .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> > > 7 files changed, 3106 insertions(+)
> > > create mode 100644 drivers/media/platform/bcm2835/Kconfig
> > > create mode 100644 drivers/media/platform/bcm2835/Makefile
> > > create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> > > create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
>
> [snip]
>
> > > diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > > new file mode 100644
> > > index 000000000000..2e9387cbc1e0
> > > --- /dev/null
> > > +++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > > @@ -0,0 +1,2825 @@
>
> [snip]
>
> > > +static int unicam_enum_frameintervals(struct file *file, void *priv,
> > > + struct v4l2_frmivalenum *fival)
> > > +{
> > > + struct unicam_node *node = video_drvdata(file);
> > > + struct unicam_device *dev = node->dev;
> > > + const struct unicam_fmt *fmt;
> > > + struct v4l2_subdev_frame_interval_enum fie = {
> > > + .index = fival->index,
> > > + .width = fival->width,
> > > + .height = fival->height,
> > > + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > > + };
> > > + int ret;
> > > +
> > > + fmt = find_format_by_pix(dev, fival->pixel_format);
> > > + if (!fmt)
> > > + return -EINVAL;
> > > +
> > > + fie.code = fmt->code;
> > > + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_interval,
> > > + NULL, &fie);
> >
> > You're adding a new CSI-2 receiver driver but your driver appears to be
> > video node centric and does not use MC / V4L2 subdev uAPIs for pipeline
> > configuration.
> >
> > This is effectively needed if you want to be able to capture embedded data.
> >
> > I'd also recommend it since this way the driver will be compliant with all
> > camera sensor drivers, not just those that expose a single sub-device.
> > There are no good ways to change this once your driver is in upstream
> > kernel.
> >
> > This is also why e.g. ipu3-cio2 driver is MC-centric.
>
> I've had lengthy discussions with Dave on this topic. While I agree with
> you in principle, Dave had good arguments for keeping this
> video-node-centric. We all agreed it wasn't a perfect solution, but it
> could still be a pragmatic one.
>
> If I remember correctly the discussion was in private e-mails though.
> Dave, I'm pretty sure you're tired of repeating the same thing, but
> Sakari can't be expected to know all we've talked about. I can try to
> summarize your points, but I may not do a very good job at defending
> your point of view given that I wish you would be wrong :-) Would you
> like to summarize your position, or should I give it a go ?
One previous thread was on libcamera-devel -
https://lists.libcamera.org/pipermail/libcamera-devel/2020-February/006789.html
The main stumbling point is the lack of userspace support for MC in
the normal applications like GStreamer and FFmpeg? Or even in simpler
apps like fswebcam or qv4l2?
AFACIT None of them can set the resolution via MC. Surely that's one
of the most fundamental operations.
The main devices I've got working are:
- ov5647 (5MPix Bayer)
- imx219 (8MPix Bayer)
- imx477 (12MPix Bayer)
- imx290/327 (1080p global shutter Bayer or mono)
- ov9281 (1MPix global shutter Bayer or mono)
- ov7251 (0.31MPix global shutter)
- tc358743 (HDMI to CSI-2 bridge)
- adv7282m (analogue video to CSI2 bridge)
None need MC for any of their functionality, and I've yet to really
see a driver that does (perhaps I've just missed them).
tc358743 & adv7282 are slightly odd in that setting the timing or
standard sets the capture resolution. Conveying that configuration
path to users is bad enough, and it isn't brilliantly supported by
apps.
For the sensor modules from a user's perspective having to invoke
media-ctl to set the resolution before starting GStreamer or FFmpeg
sucks. Why are we forcing them down that route?
If video-node-centric then a GStreamer pipeline along the lines of
gst-launch-1.0 -e v4l2src ! "video/x-raw,width=W,height=H,format=Y10P"
! v4l2convert ! "video/x-raw,width=W,height=H,format=I420" ! <some
sink>
just works and can set everything up. Same with FFmpeg.
There isn't an equivalent one-line pipeline in an MC-centric world
that can set the resolution.
libcamera starts to address that restriction, but isn't applicable for
tc358743 or adv7282, and potentially only limited use for mono sensors
(eg users want low latency to some machine vision library).
So, unless I've missed something, if we adopt MC it makes libcamera
practically mandatory for all Bayer sensors, and we force users of
those other devices into an additional API with manual configuration
as none of the apps support it.
Unicam doesn't have any significant processing stages hidden within
it, only unpacking raw formats to 16bpp which is handled via pixel
formats. Otherwise it purely takes the data from the CSI2/CCP2 bus and
writes it to SDRAM.
MC is there for the complex pipelines, but we have a simple one!
Can you be the sales-person for MC here and tell me what problem it
actually solves for the user in my case? I've had this driver kicking
around in our tree for a while, so to make the change to MC means I
need to justify it to users, and provide them the tools to do the same
as they currently can. At present I can't do that.
A quick look at the docs says the MC API has been finalised since
around 4.8 by the looks of it, so nearly 4 years. IMHO For none of the
main userspace apps to handle it says a huge amount. Sorry.
If there was a clear way of implementing sufficient MC support within
GStreamer and FFmpeg and there was someone willing to take on the
work, then I could see if I can get my bosses to pay to subcontract
the work and get it upstreamed (we don't want to be maintaining
patches to those apps). Of course that still leaves a million and one
other apps out there which won't work.
(Thinking out loud. If there is only one link in the graph that
terminates in the appropriate /dev/video node, then try setting the
resolution on that? Would that cover it? Does it cover it for the
simple cases on other hardware? Would it be acceptable to GStreamer
and FFmpeg? Have we just introduced the mapping table between MBUS
formats and V4L2_PIX_FMT_xxx into both apps? All these questions!)
> > > + if (ret)
> > > + return ret;
> > > +
> > > + fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
> > > + fival->discrete = fie.interval;
> > > +
> > > + return 0;
> > > +}
>
> [snip]
>
> > > +static int register_node(struct unicam_device *unicam, struct unicam_node *node,
> > > + enum v4l2_buf_type type, int pad_id)
> > > +{
>
> [snip]
>
> > > + if (pad_id != METADATA_PAD || unicam->sensor_embedded_data) {
> > > + ret = media_create_pad_link(&unicam->sensor->entity, pad_id,
> > > + &node->video_dev.entity, 0,
> > > + MEDIA_LNK_FL_ENABLED |
> > > + MEDIA_LNK_FL_IMMUTABLE);
> >
> > This does create two links between the sensor and the CSI-2 receiver,
> > doesn't it?
> >
> > The links in Media controller represent physical links, not logical flows
> > of data. At the time the API was added, it wasn't thought there would be a
> > need to separate the two.
> >
> > There is an effort to add the concept of data flow to V4L2, but it's been
> > complicated and we haven't had patches for the CSI-2 receiver driver to
> > support it. Perhaps Unicam could be the first one to do that?
>
> I agree that this is the right approach. The V4L2 multiplexed streams
> support seems to be one of these cursed series, where bad things happen
> to anyone who touches it. I was about to actively start working on it
> again back in June for a different project, which then got frozen at the
> last minute :-S
>
> Would you like to give it a try ? :-) I'd be more than happy to provide
> you hardware as a present.
If you want hardware then we can provide it and save Laurent's wallet.
Email me an address and phone number (our couriers now insist on it)
and I can get some sorted.
> > Alternatively support for embedded data could be removed in the meantime.
If that's what it takes, then OK, but using embedded data removes the
guesswork from knowing the gain and exposures actually used, and so
avoids oscillations. Some sensors don't support embedded data so you
have to guess, but it's a shame if it is available and it can't be
used.
I'm between a rock and a hard place:
- Libcamera wants platforms to be supported on mainline kernels (or at
least submitted to the list)
- But there isn't a framework available to let us do what is needed,
and I don't know enough of the history and use cases to really work on
it directly.
How to satisfy everyone? :-/
Alternatively I end up with a driver that has a flag to switch between
MC-centric and video-device-centric modes (I would say a DT flag, but
would it get past the DT maintainers as it isn't describing the
hardware?)
- If you want libcamera and embedded data then go MC and you have to
jump through hoops to set up resolution.
- If you don't want libcamera or embedded data then go video-centric
and use all the tools you're used to.
Possible, but there are going to be so many conditionals dotted around
it would get ugly.
Enough rambling from me. Thanks for your time in reviewing this lot -
hopefully we can find a way forward.
Dave
> > The latest patchset is here I believe:
> >
> > <URL:https://patchwork.kernel.org/project/linux-media/list/?series=98277>
> >
> > > + if (ret)
> > > + unicam_err(unicam, "Unable to create pad link for %s\n",
> > > + vdev->name);
> > > + }
> > > +
> > > + return ret;
> > > +}
>
> --
> Regards,
>
> Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-09-15 13:28 ` Dave Stevenson
@ 2020-10-30 17:53 ` Jacopo Mondi
0 siblings, 0 replies; 111+ messages in thread
From: Jacopo Mondi @ 2020-10-30 17:53 UTC (permalink / raw)
To: Dave Stevenson
Cc: Laurent Pinchart, Sakari Ailus, Linux Media Mailing List,
Kieran Bingham, Niklas Söderlund, Naushir Patuck
Hello Sakari, Dave, Laurent,
On Tue, Sep 15, 2020 at 02:28:16PM +0100, Dave Stevenson wrote:
> Hi Sakari & Laurent
>
> On Tue, 15 Sep 2020 at 10:33, Laurent Pinchart
> <laurent.pinchart@ideasonboard.com> wrote:
> >
> > Hi Sakari,
> >
> > (With a question for Dave below)
> >
> > I'm replying to the two main points of your review. All the other
> > comments look fine at a glance, Jacopo is having a more detailed look
> > and will incorporate them in v3.
> >
> > On Tue, Sep 15, 2020 at 10:03:26AM +0300, Sakari Ailus wrote:
> > > Hi Laurent,
> > >
> > > Thanks for the patch, and my apologies for the late review. Please do cc me
> > > for v3.
> > >
> > > After a quick look I can already say this is the cleanest Unicam driver
> > > I've ever seen. But please also see my detailed comments below.
> > >
> > > On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > > > From: Naushir Patuck <naush@raspberrypi.com>
> > > >
> > > > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > > > Compared to the bcm2835-camera driver present in staging, this driver
> > > > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > > > the VC4 firmware running on the VPU.
> > > >
> > > > The commit is made up of a series of changes cherry-picked from the
> > > > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > > > additional enhancements, forward-ported to the mainline kernel.
> > > >
> > > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > > > ---
> > > > Changes since v1:
> > > >
> > > > - Re-fetch mbus code from subdev on a g_fmt call
> > > > - Group all ioctl disabling together
> > > > - Fix reference counting in unicam_open
> > > > - Add support for VIDIOC_[S|G]_SELECTION
> > > > ---
> > > > MAINTAINERS | 7 +
> > > > drivers/media/platform/Kconfig | 1 +
> > > > drivers/media/platform/Makefile | 2 +
> > > > drivers/media/platform/bcm2835/Kconfig | 15 +
> > > > drivers/media/platform/bcm2835/Makefile | 3 +
> > > > .../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
> > > > .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> > > > 7 files changed, 3106 insertions(+)
> > > > create mode 100644 drivers/media/platform/bcm2835/Kconfig
> > > > create mode 100644 drivers/media/platform/bcm2835/Makefile
> > > > create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> > > > create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
> >
> > [snip]
> >
> > > > diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > > > new file mode 100644
> > > > index 000000000000..2e9387cbc1e0
> > > > --- /dev/null
> > > > +++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > > > @@ -0,0 +1,2825 @@
> >
> > [snip]
> >
> > > > +static int unicam_enum_frameintervals(struct file *file, void *priv,
> > > > + struct v4l2_frmivalenum *fival)
> > > > +{
> > > > + struct unicam_node *node = video_drvdata(file);
> > > > + struct unicam_device *dev = node->dev;
> > > > + const struct unicam_fmt *fmt;
> > > > + struct v4l2_subdev_frame_interval_enum fie = {
> > > > + .index = fival->index,
> > > > + .width = fival->width,
> > > > + .height = fival->height,
> > > > + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > > > + };
> > > > + int ret;
> > > > +
> > > > + fmt = find_format_by_pix(dev, fival->pixel_format);
> > > > + if (!fmt)
> > > > + return -EINVAL;
> > > > +
> > > > + fie.code = fmt->code;
> > > > + ret = v4l2_subdev_call(dev->sensor, pad, enum_frame_interval,
> > > > + NULL, &fie);
> > >
> > > You're adding a new CSI-2 receiver driver but your driver appears to be
> > > video node centric and does not use MC / V4L2 subdev uAPIs for pipeline
> > > configuration.
> > >
> > > This is effectively needed if you want to be able to capture embedded data.
> > >
> > > I'd also recommend it since this way the driver will be compliant with all
> > > camera sensor drivers, not just those that expose a single sub-device.
> > > There are no good ways to change this once your driver is in upstream
> > > kernel.
> > >
> > > This is also why e.g. ipu3-cio2 driver is MC-centric.
> >
> > I've had lengthy discussions with Dave on this topic. While I agree with
> > you in principle, Dave had good arguments for keeping this
> > video-node-centric. We all agreed it wasn't a perfect solution, but it
> > could still be a pragmatic one.
> >
> > If I remember correctly the discussion was in private e-mails though.
> > Dave, I'm pretty sure you're tired of repeating the same thing, but
> > Sakari can't be expected to know all we've talked about. I can try to
> > summarize your points, but I may not do a very good job at defending
> > your point of view given that I wish you would be wrong :-) Would you
> > like to summarize your position, or should I give it a go ?
>
> One previous thread was on libcamera-devel -
> https://lists.libcamera.org/pipermail/libcamera-devel/2020-February/006789.html
>
> The main stumbling point is the lack of userspace support for MC in
> the normal applications like GStreamer and FFmpeg? Or even in simpler
> apps like fswebcam or qv4l2?
> AFACIT None of them can set the resolution via MC. Surely that's one
> of the most fundamental operations.
>
> The main devices I've got working are:
> - ov5647 (5MPix Bayer)
> - imx219 (8MPix Bayer)
> - imx477 (12MPix Bayer)
> - imx290/327 (1080p global shutter Bayer or mono)
> - ov9281 (1MPix global shutter Bayer or mono)
> - ov7251 (0.31MPix global shutter)
> - tc358743 (HDMI to CSI-2 bridge)
> - adv7282m (analogue video to CSI2 bridge)
>
> None need MC for any of their functionality, and I've yet to really
> see a driver that does (perhaps I've just missed them).
>
> tc358743 & adv7282 are slightly odd in that setting the timing or
> standard sets the capture resolution. Conveying that configuration
> path to users is bad enough, and it isn't brilliantly supported by
> apps.
>
> For the sensor modules from a user's perspective having to invoke
> media-ctl to set the resolution before starting GStreamer or FFmpeg
> sucks. Why are we forcing them down that route?
> If video-node-centric then a GStreamer pipeline along the lines of
> gst-launch-1.0 -e v4l2src ! "video/x-raw,width=W,height=H,format=Y10P"
> ! v4l2convert ! "video/x-raw,width=W,height=H,format=I420" ! <some
> sink>
> just works and can set everything up. Same with FFmpeg.
> There isn't an equivalent one-line pipeline in an MC-centric world
> that can set the resolution.
>
> libcamera starts to address that restriction, but isn't applicable for
> tc358743 or adv7282, and potentially only limited use for mono sensors
> (eg users want low latency to some machine vision library).
> So, unless I've missed something, if we adopt MC it makes libcamera
> practically mandatory for all Bayer sensors, and we force users of
> those other devices into an additional API with manual configuration
> as none of the apps support it.
>
> Unicam doesn't have any significant processing stages hidden within
> it, only unpacking raw formats to 16bpp which is handled via pixel
> formats. Otherwise it purely takes the data from the CSI2/CCP2 bus and
> writes it to SDRAM.
> MC is there for the complex pipelines, but we have a simple one!
>
>
> Can you be the sales-person for MC here and tell me what problem it
> actually solves for the user in my case? I've had this driver kicking
> around in our tree for a while, so to make the change to MC means I
> need to justify it to users, and provide them the tools to do the same
> as they currently can. At present I can't do that.
> A quick look at the docs says the MC API has been finalised since
> around 4.8 by the looks of it, so nearly 4 years. IMHO For none of the
> main userspace apps to handle it says a huge amount. Sorry.
>
>
> If there was a clear way of implementing sufficient MC support within
> GStreamer and FFmpeg and there was someone willing to take on the
> work, then I could see if I can get my bosses to pay to subcontract
> the work and get it upstreamed (we don't want to be maintaining
> patches to those apps). Of course that still leaves a million and one
> other apps out there which won't work.
>
> (Thinking out loud. If there is only one link in the graph that
> terminates in the appropriate /dev/video node, then try setting the
> resolution on that? Would that cover it? Does it cover it for the
> simple cases on other hardware? Would it be acceptable to GStreamer
> and FFmpeg? Have we just introduced the mapping table between MBUS
> formats and V4L2_PIX_FMT_xxx into both apps? All these questions!)
>
> > > > + if (ret)
> > > > + return ret;
> > > > +
> > > > + fival->type = V4L2_FRMIVAL_TYPE_DISCRETE;
> > > > + fival->discrete = fie.interval;
> > > > +
> > > > + return 0;
> > > > +}
> >
> > [snip]
> >
> > > > +static int register_node(struct unicam_device *unicam, struct unicam_node *node,
> > > > + enum v4l2_buf_type type, int pad_id)
> > > > +{
> >
> > [snip]
> >
> > > > + if (pad_id != METADATA_PAD || unicam->sensor_embedded_data) {
> > > > + ret = media_create_pad_link(&unicam->sensor->entity, pad_id,
> > > > + &node->video_dev.entity, 0,
> > > > + MEDIA_LNK_FL_ENABLED |
> > > > + MEDIA_LNK_FL_IMMUTABLE);
> > >
> > > This does create two links between the sensor and the CSI-2 receiver,
> > > doesn't it?
> > >
> > > The links in Media controller represent physical links, not logical flows
> > > of data. At the time the API was added, it wasn't thought there would be a
> > > need to separate the two.
> > >
> > > There is an effort to add the concept of data flow to V4L2, but it's been
> > > complicated and we haven't had patches for the CSI-2 receiver driver to
> > > support it. Perhaps Unicam could be the first one to do that?
> >
> > I agree that this is the right approach. The V4L2 multiplexed streams
> > support seems to be one of these cursed series, where bad things happen
> > to anyone who touches it. I was about to actively start working on it
> > again back in June for a different project, which then got frozen at the
> > last minute :-S
> >
> > Would you like to give it a try ? :-) I'd be more than happy to provide
> > you hardware as a present.
>
> If you want hardware then we can provide it and save Laurent's wallet.
> Email me an address and phone number (our couriers now insist on it)
> and I can get some sorted.
>
> > > Alternatively support for embedded data could be removed in the meantime.
>
> If that's what it takes, then OK, but using embedded data removes the
> guesswork from knowing the gain and exposures actually used, and so
> avoids oscillations. Some sensors don't support embedded data so you
> have to guess, but it's a shame if it is available and it can't be
> used.
>
> I'm between a rock and a hard place:
> - Libcamera wants platforms to be supported on mainline kernels (or at
> least submitted to the list)
> - But there isn't a framework available to let us do what is needed,
> and I don't know enough of the history and use cases to really work on
> it directly.
> How to satisfy everyone? :-/
>
> Alternatively I end up with a driver that has a flag to switch between
> MC-centric and video-device-centric modes (I would say a DT flag, but
> would it get past the DT maintainers as it isn't describing the
> hardware?)
> - If you want libcamera and embedded data then go MC and you have to
> jump through hoops to set up resolution.
> - If you don't want libcamera or embedded data then go video-centric
> and use all the tools you're used to.
> Possible, but there are going to be so many conditionals dotted around
> it would get ugly.
>
> Enough rambling from me. Thanks for your time in reviewing this lot -
> hopefully we can find a way forward.
>
> Dave
I would be pleased to see this discussion continue, but in the
meantime, not to block v3, I'll move the driver to staging if no one
objects.
Thanks
j
>
> > > The latest patchset is here I believe:
> > >
> > > <URL:https://patchwork.kernel.org/project/linux-media/list/?series=98277>
> > >
> > > > + if (ret)
> > > > + unicam_err(unicam, "Unable to create pad link for %s\n",
> > > > + vdev->name);
> > > > + }
> > > > +
> > > > + return ret;
> > > > +}
> >
> > --
> > Regards,
> >
> > Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
2020-09-15 7:03 ` Sakari Ailus
2020-09-15 9:32 ` Laurent Pinchart
@ 2020-09-15 17:30 ` Dave Stevenson
1 sibling, 0 replies; 111+ messages in thread
From: Dave Stevenson @ 2020-09-15 17:30 UTC (permalink / raw)
To: Sakari Ailus
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
Hi Sakari
Answering one of the review comments
On Tue, 15 Sep 2020 at 08:04, Sakari Ailus <sakari.ailus@iki.fi> wrote:
>
> Hi Laurent,
>
> Thanks for the patch, and my apologies for the late review. Please do cc me
> for v3.
>
> After a quick look I can already say this is the cleanest Unicam driver
> I've ever seen. But please also see my detailed comments below.
>
> On Mon, May 04, 2020 at 12:25:41PM +0300, Laurent Pinchart wrote:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Add a driver for the Unicam camera receiver block on BCM283x processors.
> > Compared to the bcm2835-camera driver present in staging, this driver
> > handles the Unicam block only (CSI-2 receiver), and doesn't depend on
> > the VC4 firmware running on the VPU.
> >
> > The commit is made up of a series of changes cherry-picked from the
> > rpi-5.4.y branch of https://github.com/raspberrypi/linux/ with
> > additional enhancements, forward-ported to the mainline kernel.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> > ---
> > Changes since v1:
> >
> > - Re-fetch mbus code from subdev on a g_fmt call
> > - Group all ioctl disabling together
> > - Fix reference counting in unicam_open
> > - Add support for VIDIOC_[S|G]_SELECTION
> > ---
> > MAINTAINERS | 7 +
> > drivers/media/platform/Kconfig | 1 +
> > drivers/media/platform/Makefile | 2 +
> > drivers/media/platform/bcm2835/Kconfig | 15 +
> > drivers/media/platform/bcm2835/Makefile | 3 +
> > .../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
> > .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> > 7 files changed, 3106 insertions(+)
> > create mode 100644 drivers/media/platform/bcm2835/Kconfig
> > create mode 100644 drivers/media/platform/bcm2835/Makefile
> > create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> > create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index b816a453b10e..edf5b8d9c98a 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -3341,6 +3341,13 @@ N: bcm113*
> > N: bcm216*
> > N: kona
> >
> > +BROADCOM BCM2835 CAMERA DRIVER
> > +M: Raspberry Pi Kernel Maintenance <kernel-list@raspberrypi.com>
> > +L: linux-media@vger.kernel.org
> > +S: Maintained
> > +F: drivers/media/platform/bcm2835/
> > +F: Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml
> > +
> > BROADCOM BCM47XX MIPS ARCHITECTURE
> > M: Hauke Mehrtens <hauke@hauke-m.de>
> > M: Rafał Miłecki <zajec5@gmail.com>
> > diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig
> > index e01bbb9dd1c1..98721a4e0be1 100644
> > --- a/drivers/media/platform/Kconfig
> > +++ b/drivers/media/platform/Kconfig
> > @@ -146,6 +146,7 @@ source "drivers/media/platform/am437x/Kconfig"
> > source "drivers/media/platform/xilinx/Kconfig"
> > source "drivers/media/platform/rcar-vin/Kconfig"
> > source "drivers/media/platform/atmel/Kconfig"
> > +source "drivers/media/platform/bcm2835/Kconfig"
> > source "drivers/media/platform/sunxi/Kconfig"
> >
> > config VIDEO_TI_CAL
> > diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile
> > index d13db96e3015..a425e4d2e3f3 100644
> > --- a/drivers/media/platform/Makefile
> > +++ b/drivers/media/platform/Makefile
> > @@ -98,4 +98,6 @@ obj-y += meson/
> >
> > obj-y += cros-ec-cec/
> >
> > +obj-y += bcm2835/
> > +
> > obj-y += sunxi/
> > diff --git a/drivers/media/platform/bcm2835/Kconfig b/drivers/media/platform/bcm2835/Kconfig
> > new file mode 100644
> > index 000000000000..ec46e3ef053c
> > --- /dev/null
> > +++ b/drivers/media/platform/bcm2835/Kconfig
> > @@ -0,0 +1,15 @@
> > +# Broadcom VideoCore4 V4L2 camera support
> > +
> > +config VIDEO_BCM2835_UNICAM
> > + tristate "Broadcom BCM2835 Unicam video capture driver"
> > + depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API && MEDIA_CONTROLLER
> > + depends on ARCH_BCM2835 || COMPILE_TEST
> > + select VIDEOBUF2_DMA_CONTIG
> > + select V4L2_FWNODE
> > + help
> > + Say Y here to enable support for the BCM2835 CSI-2 receiver. This is a
> > + V4L2 driver that controls the CSI-2 receiver directly, independently
> > + from the VC4 firmware.
>
> \o/
>
> > +
> > + To compile this driver as a module, choose M here. The module will be
> > + called bcm2835-unicam.
> > diff --git a/drivers/media/platform/bcm2835/Makefile b/drivers/media/platform/bcm2835/Makefile
> > new file mode 100644
> > index 000000000000..a98aba03598a
> > --- /dev/null
> > +++ b/drivers/media/platform/bcm2835/Makefile
> > @@ -0,0 +1,3 @@
> > +# Makefile for BCM2835 Unicam driver
> > +
> > +obj-$(CONFIG_VIDEO_BCM2835_UNICAM) += bcm2835-unicam.o
> > diff --git a/drivers/media/platform/bcm2835/bcm2835-unicam.c b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > new file mode 100644
> > index 000000000000..2e9387cbc1e0
> > --- /dev/null
> > +++ b/drivers/media/platform/bcm2835/bcm2835-unicam.c
> > @@ -0,0 +1,2825 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/*
> > + * BCM2835 Unicam Capture Driver
> > + *
> > + * Copyright (C) 2017-2020 - Raspberry Pi (Trading) Ltd.
> > + *
> > + * Dave Stevenson <dave.stevenson@raspberrypi.com>
> > + *
> > + * Based on TI am437x driver by
> > + * Benoit Parrot <bparrot@ti.com>
> > + * Lad, Prabhakar <prabhakar.csengg@gmail.com>
> > + *
> > + * and TI CAL camera interface driver by
> > + * Benoit Parrot <bparrot@ti.com>
> > + *
> > + *
> > + * There are two camera drivers in the kernel for BCM283x - this one
> > + * and bcm2835-camera (currently in staging).
> > + *
> > + * This driver directly controls the Unicam peripheral - there is no
> > + * involvement with the VideoCore firmware. Unicam receives CSI-2 or
> > + * CCP2 data and writes it into SDRAM.
> > + * The only potential processing options are to repack Bayer data into an
> > + * alternate format, and applying windowing.
> > + * The repacking does not shift the data, so can repack V4L2_PIX_FMT_Sxxxx10P
> > + * to V4L2_PIX_FMT_Sxxxx10, or V4L2_PIX_FMT_Sxxxx12P to V4L2_PIX_FMT_Sxxxx12,
> > + * but not generically up to V4L2_PIX_FMT_Sxxxx16. The driver will add both
> > + * formats where the relevant formats are defined, and will automatically
> > + * configure the repacking as required.
> > + * Support for windowing may be added later.
> > + *
> > + * It should be possible to connect this driver to any sensor with a
> > + * suitable output interface and V4L2 subdevice driver.
> > + *
> > + * bcm2835-camera uses the VideoCore firmware to control the sensor,
> > + * Unicam, ISP, and all tuner control loops. Fully processed frames are
> > + * delivered to the driver by the firmware. It only has sensor drivers
> > + * for Omnivision OV5647, and Sony IMX219 sensors.
> > + *
> > + * The two drivers are mutually exclusive for the same Unicam instance.
> > + * The VideoCore firmware checks the device tree configuration during boot.
> > + * If it finds device tree nodes called csi0 or csi1 it will block the
> > + * firmware from accessing the peripheral, and bcm2835-camera will
> > + * not be able to stream data.
> > + */
> > +
> > +#include <linux/clk.h>
> > +#include <linux/delay.h>
> > +#include <linux/device.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/err.h>
> > +#include <linux/init.h>
> > +#include <linux/interrupt.h>
> > +#include <linux/io.h>
> > +#include <linux/module.h>
> > +#include <linux/of_device.h>
> > +#include <linux/of_graph.h>
> > +#include <linux/pinctrl/consumer.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/pm_runtime.h>
> > +#include <linux/slab.h>
> > +#include <linux/uaccess.h>
> > +#include <linux/videodev2.h>
> > +
> > +#include <media/v4l2-common.h>
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-dev.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-dv-timings.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/v4l2-fwnode.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "vc4-regs-unicam.h"
> > +
> > +#define UNICAM_MODULE_NAME "unicam"
> > +#define UNICAM_VERSION "0.1.0"
> > +
> > +static int debug;
> > +module_param(debug, int, 0644);
> > +MODULE_PARM_DESC(debug, "Debug level 0-3");
> > +
> > +#define unicam_dbg(level, dev, fmt, arg...) \
> > + v4l2_dbg(level, debug, &(dev)->v4l2_dev, fmt, ##arg)
> > +#define unicam_info(dev, fmt, arg...) \
> > + v4l2_info(&(dev)->v4l2_dev, fmt, ##arg)
> > +#define unicam_err(dev, fmt, arg...) \
> > + v4l2_err(&(dev)->v4l2_dev, fmt, ##arg)
>
> I'd just use dev_*() macros in new drivers unless you really need something
> else. It's configurable. Up to you...
>
> > +
> > +/*
> > + * To protect against a dodgy sensor driver never returning an error from
> > + * enum_mbus_code, set a maximum index value to be used.
> > + */
> > +#define MAX_ENUM_MBUS_CODE 128
>
> Hmm.
>
> Is it Unicam driver's job to work around sensor driver bugs? That same bug
> would hit user space, too. I'd say such a sensor driver must be fixed, and
> not just worked around here and there.
Yes, it would hit userspace too.
However the issue from Unicam's side is that in order to support and
enumerate both the packed and unpacked version of each Bayer or mono
format (eg RGGB10 and RGGB10P), unicam_enum_fmt_vid_cap loops until
either it reaches the requested index (and it only counts supported
formats), or the subdev returns an error. If the subdev has a bug such
that it doesn't check index and return an error, and the last format
isn't supported, then there is no way of breaking that loop. Being
stuck in a loop within the kernel, whether the fault of the subdev or
Unicam, is surely a bad thing, hence some defensive programming.
Yes, I was bitten by this, but I don't recall which driver it was that
I was working on at the time.
> > +
> > +/*
> > + * Stride is a 16 bit register, but also has to be a multiple of 32.
> > + */
> > +#define BPL_ALIGNMENT 32
> > +#define MAX_BYTESPERLINE ((1 << 16) - BPL_ALIGNMENT)
> > +/*
> > + * Max width is therefore determined by the max stride divided by
> > + * the number of bits per pixel. Take 32bpp as a
> > + * worst case.
> > + * No imposed limit on the height, so adopt a square image for want
> > + * of anything better.
> > + */
> > +#define MAX_WIDTH (MAX_BYTESPERLINE / 4)
> > +#define MAX_HEIGHT MAX_WIDTH
> > +/* Define a nominal minimum image size */
> > +#define MIN_WIDTH 16
> > +#define MIN_HEIGHT 16
> > +/* Default size of the embedded buffer */
> > +#define UNICAM_EMBEDDED_SIZE 8192
> > +
> > +/*
> > + * Size of the dummy buffer. Can be any size really, but the DMA
> > + * allocation works in units of page sizes.
> > + */
> > +#define DUMMY_BUF_SIZE (PAGE_SIZE)
> > +
> > +enum pad_types {
> > + IMAGE_PAD,
> > + METADATA_PAD,
> > + MAX_NODES
> > +};
> > +
> > +/*
> > + * struct unicam_fmt - Unicam media bus format information
> > + * @pixelformat: V4L2 pixel format FCC identifier. 0 if n/a.
> > + * @repacked_fourcc: V4L2 pixel format FCC identifier if the data is expanded
> > + * out to 16bpp. 0 if n/a.
> > + * @code: V4L2 media bus format code.
> > + * @depth: Bits per pixel as delivered from the source.
> > + * @csi_dt: CSI data type.
> > + * @check_variants: Flag to denote that there are multiple mediabus formats
> > + * still in the list that could match this V4L2 format.
> > + */
> > +struct unicam_fmt {
> > + u32 fourcc;
> > + u32 repacked_fourcc;
> > + u32 code;
> > + u8 depth;
> > + u8 csi_dt;
> > + u8 check_variants;
> > +};
> > +
> > +static const struct unicam_fmt formats[] = {
> > + /* YUV Formats */
> > + {
> > + .fourcc = V4L2_PIX_FMT_YUYV,
> > + .code = MEDIA_BUS_FMT_YUYV8_2X8,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + .check_variants = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_UYVY,
> > + .code = MEDIA_BUS_FMT_UYVY8_2X8,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + .check_variants = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YVYU,
> > + .code = MEDIA_BUS_FMT_YVYU8_2X8,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + .check_variants = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_VYUY,
> > + .code = MEDIA_BUS_FMT_VYUY8_2X8,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + .check_variants = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YUYV,
> > + .code = MEDIA_BUS_FMT_YUYV8_1X16,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_UYVY,
> > + .code = MEDIA_BUS_FMT_UYVY8_1X16,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YVYU,
> > + .code = MEDIA_BUS_FMT_YVYU8_1X16,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_VYUY,
> > + .code = MEDIA_BUS_FMT_VYUY8_1X16,
> > + .depth = 16,
> > + .csi_dt = 0x1e,
> > + }, {
> > + /* RGB Formats */
> > + .fourcc = V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
> > + .code = MEDIA_BUS_FMT_RGB565_2X8_LE,
> > + .depth = 16,
> > + .csi_dt = 0x22,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_RGB565X, /* rrrrrggg gggbbbbb */
> > + .code = MEDIA_BUS_FMT_RGB565_2X8_BE,
> > + .depth = 16,
> > + .csi_dt = 0x22
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_RGB555, /* gggbbbbb arrrrrgg */
> > + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_LE,
> > + .depth = 16,
> > + .csi_dt = 0x21,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_RGB555X, /* arrrrrgg gggbbbbb */
> > + .code = MEDIA_BUS_FMT_RGB555_2X8_PADHI_BE,
> > + .depth = 16,
> > + .csi_dt = 0x21,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_RGB24, /* rgb */
> > + .code = MEDIA_BUS_FMT_RGB888_1X24,
> > + .depth = 24,
> > + .csi_dt = 0x24,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_BGR24, /* bgr */
> > + .code = MEDIA_BUS_FMT_BGR888_1X24,
> > + .depth = 24,
> > + .csi_dt = 0x24,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_RGB32, /* argb */
> > + .code = MEDIA_BUS_FMT_ARGB8888_1X32,
> > + .depth = 32,
> > + .csi_dt = 0x0,
> > + }, {
> > + /* Bayer Formats */
> > + .fourcc = V4L2_PIX_FMT_SBGGR8,
> > + .code = MEDIA_BUS_FMT_SBGGR8_1X8,
> > + .depth = 8,
> > + .csi_dt = 0x2a,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG8,
> > + .code = MEDIA_BUS_FMT_SGBRG8_1X8,
> > + .depth = 8,
> > + .csi_dt = 0x2a,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG8,
> > + .code = MEDIA_BUS_FMT_SGRBG8_1X8,
> > + .depth = 8,
> > + .csi_dt = 0x2a,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SRGGB8,
> > + .code = MEDIA_BUS_FMT_SRGGB8_1X8,
> > + .depth = 8,
> > + .csi_dt = 0x2a,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR10P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SBGGR10,
> > + .code = MEDIA_BUS_FMT_SBGGR10_1X10,
> > + .depth = 10,
> > + .csi_dt = 0x2b,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG10P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SGBRG10,
> > + .code = MEDIA_BUS_FMT_SGBRG10_1X10,
> > + .depth = 10,
> > + .csi_dt = 0x2b,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG10P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SGRBG10,
> > + .code = MEDIA_BUS_FMT_SGRBG10_1X10,
> > + .depth = 10,
> > + .csi_dt = 0x2b,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SRGGB10P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SRGGB10,
> > + .code = MEDIA_BUS_FMT_SRGGB10_1X10,
> > + .depth = 10,
> > + .csi_dt = 0x2b,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR12P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SBGGR12,
> > + .code = MEDIA_BUS_FMT_SBGGR12_1X12,
> > + .depth = 12,
> > + .csi_dt = 0x2c,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG12P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SGBRG12,
> > + .code = MEDIA_BUS_FMT_SGBRG12_1X12,
> > + .depth = 12,
> > + .csi_dt = 0x2c,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG12P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SGRBG12,
> > + .code = MEDIA_BUS_FMT_SGRBG12_1X12,
> > + .depth = 12,
> > + .csi_dt = 0x2c,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SRGGB12P,
> > + .repacked_fourcc = V4L2_PIX_FMT_SRGGB12,
> > + .code = MEDIA_BUS_FMT_SRGGB12_1X12,
> > + .depth = 12,
> > + .csi_dt = 0x2c,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR14P,
> > + .code = MEDIA_BUS_FMT_SBGGR14_1X14,
> > + .depth = 14,
> > + .csi_dt = 0x2d,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG14P,
> > + .code = MEDIA_BUS_FMT_SGBRG14_1X14,
> > + .depth = 14,
> > + .csi_dt = 0x2d,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG14P,
> > + .code = MEDIA_BUS_FMT_SGRBG14_1X14,
> > + .depth = 14,
> > + .csi_dt = 0x2d,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SRGGB14P,
> > + .code = MEDIA_BUS_FMT_SRGGB14_1X14,
> > + .depth = 14,
> > + .csi_dt = 0x2d,
> > + }, {
> > + /*
> > + * 16 bit Bayer formats could be supported, but there is no CSI2
> > + * data_type defined for raw 16, and no sensors that produce it at
> > + * present.
> > + */
> > +
> > + /* Greyscale formats */
> > + .fourcc = V4L2_PIX_FMT_GREY,
> > + .code = MEDIA_BUS_FMT_Y8_1X8,
> > + .depth = 8,
> > + .csi_dt = 0x2a,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_Y10P,
> > + .repacked_fourcc = V4L2_PIX_FMT_Y10,
> > + .code = MEDIA_BUS_FMT_Y10_1X10,
> > + .depth = 10,
> > + .csi_dt = 0x2b,
> > + }, {
> > + /* NB There is no packed V4L2 fourcc for this format. */
> > + .repacked_fourcc = V4L2_PIX_FMT_Y12,
> > + .code = MEDIA_BUS_FMT_Y12_1X12,
> > + .depth = 12,
> > + .csi_dt = 0x2c,
> > + },
> > + /* Embedded data format */
> > + {
> > + .fourcc = V4L2_META_FMT_SENSOR_DATA,
> > + .code = MEDIA_BUS_FMT_SENSOR_DATA,
> > + .depth = 8,
> > + }
> > +};
> > +
> > +struct unicam_buffer {
> > + struct vb2_v4l2_buffer vb;
> > + struct list_head list;
> > +};
> > +
> > +static inline struct unicam_buffer *to_unicam_buffer(struct vb2_buffer *vb)
> > +{
> > + return container_of(vb, struct unicam_buffer, vb.vb2_buf);
> > +}
> > +
> > +struct unicam_node {
> > + bool registered;
> > + int open;
> > + bool streaming;
> > + unsigned int pad_id;
> > + /* Pointer pointing to current v4l2_buffer */
> > + struct unicam_buffer *cur_frm;
> > + /* Pointer pointing to next v4l2_buffer */
> > + struct unicam_buffer *next_frm;
> > + /* video capture */
> > + const struct unicam_fmt *fmt;
> > + /* Used to store current pixel format */
> > + struct v4l2_format v_fmt;
> > + /* Used to store current mbus frame format */
> > + struct v4l2_mbus_framefmt m_fmt;
> > + /* Buffer queue used in video-buf */
> > + struct vb2_queue buffer_queue;
> > + /* Queue of filled frames */
> > + struct list_head dma_queue;
> > + /* IRQ lock for DMA queue */
> > + spinlock_t dma_queue_lock;
> > + /* lock used to access this structure */
> > + struct mutex lock;
> > + /* Identifies video device for this channel */
> > + struct video_device video_dev;
> > + /* Pointer to the parent handle */
> > + struct unicam_device *dev;
> > + struct media_pad pad;
> > + unsigned int embedded_lines;
> > + /*
> > + * Dummy buffer intended to be used by unicam
> > + * if we have no other queued buffers to swap to.
> > + */
> > + void *dummy_buf_cpu_addr;
> > + dma_addr_t dummy_buf_dma_addr;
> > +};
> > +
> > +struct unicam_device {
> > + struct kref kref;
> > +
> > + /* V4l2 specific parameters */
> > + struct v4l2_async_subdev asd;
> > +
> > + /* peripheral base address */
> > + void __iomem *base;
> > + /* clock gating base address */
> > + void __iomem *clk_gate_base;
> > + /* clock handle */
> > + struct clk *clock;
> > + /* V4l2 device */
> > + struct v4l2_device v4l2_dev;
> > + struct media_device mdev;
> > +
> > + /* parent device */
> > + struct platform_device *pdev;
> > + /* subdevice async Notifier */
> > + struct v4l2_async_notifier notifier;
> > + unsigned int sequence;
> > +
> > + /* ptr to sub device */
> > + struct v4l2_subdev *sensor;
> > + /* Pad config for the sensor */
> > + struct v4l2_subdev_pad_config *sensor_config;
> > +
> > + enum v4l2_mbus_type bus_type;
> > + /*
> > + * Stores bus.mipi_csi2.flags for CSI2 sensors, or
> > + * bus.mipi_csi1.strobe for CCP2.
> > + */
> > + unsigned int bus_flags;
> > + unsigned int max_data_lanes;
> > + unsigned int active_data_lanes;
> > + bool sensor_embedded_data;
> > +
> > + struct unicam_node node[MAX_NODES];
> > + struct v4l2_ctrl_handler ctrl_handler;
> > +};
> > +
> > +static inline struct unicam_device *
> > +to_unicam_device(struct v4l2_device *v4l2_dev)
> > +{
> > + return container_of(v4l2_dev, struct unicam_device, v4l2_dev);
> > +}
> > +
> > +/* Hardware access */
> > +static inline void clk_write(struct unicam_device *dev, u32 val)
> > +{
> > + writel(val | 0x5a000000, dev->clk_gate_base);
> > +}
> > +
> > +static inline u32 reg_read(struct unicam_device *dev, u32 offset)
> > +{
> > + return readl(dev->base + offset);
> > +}
> > +
> > +static inline void reg_write(struct unicam_device *dev, u32 offset, u32 val)
> > +{
> > + writel(val, dev->base + offset);
> > +}
> > +
> > +static inline int get_field(u32 value, u32 mask)
> > +{
> > + return (value & mask) >> __ffs(mask);
> > +}
> > +
> > +static inline void set_field(u32 *valp, u32 field, u32 mask)
> > +{
> > + u32 val = *valp;
> > +
> > + val &= ~mask;
> > + val |= (field << __ffs(mask)) & mask;
> > + *valp = val;
> > +}
> > +
> > +static inline u32 reg_read_field(struct unicam_device *dev, u32 offset,
> > + u32 mask)
> > +{
> > + return get_field(reg_read(dev, offset), mask);
> > +}
> > +
> > +static inline void reg_write_field(struct unicam_device *dev, u32 offset,
> > + u32 field, u32 mask)
> > +{
> > + u32 val = reg_read(dev, offset);
> > +
> > + set_field(&val, field, mask);
> > + reg_write(dev, offset, val);
> > +}
> > +
> > +/* Power management functions */
> > +static inline int unicam_runtime_get(struct unicam_device *dev)
> > +{
> > + return pm_runtime_get_sync(&dev->pdev->dev);
> > +}
> > +
> > +static inline void unicam_runtime_put(struct unicam_device *dev)
> > +{
> > + pm_runtime_put_sync(&dev->pdev->dev);
> > +}
>
> Please don't add such wrappers.
>
> > +
> > +/* Format setup functions */
> > +static const struct unicam_fmt *find_format_by_code(u32 code)
> > +{
> > + unsigned int i;
> > +
> > + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> > + if (formats[i].code == code)
> > + return &formats[i];
> > + }
> > +
> > + return NULL;
> > +}
> > +
> > +static int check_mbus_format(struct unicam_device *dev,
> > + const struct unicam_fmt *format)
> > +{
> > + unsigned int i;
> > + int ret = 0;
> > +
> > + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> > + struct v4l2_subdev_mbus_code_enum mbus_code = {
> > + .index = i,
> > + .pad = IMAGE_PAD,
> > + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > + };
> > +
> > + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
> > + NULL, &mbus_code);
> > +
> > + if (!ret && mbus_code.code == format->code)
> > + return 1;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static const struct unicam_fmt *find_format_by_pix(struct unicam_device *dev,
> > + u32 pixelformat)
> > +{
> > + unsigned int i;
> > +
> > + for (i = 0; i < ARRAY_SIZE(formats); i++) {
> > + if (formats[i].fourcc == pixelformat ||
> > + formats[i].repacked_fourcc == pixelformat) {
> > + if (formats[i].check_variants &&
> > + !check_mbus_format(dev, &formats[i]))
> > + continue;
> > + return &formats[i];
> > + }
> > + }
> > +
> > + return NULL;
> > +}
> > +
> > +static inline unsigned int bytes_per_line(u32 width,
> > + const struct unicam_fmt *fmt,
> > + u32 v4l2_fourcc)
> > +{
> > + if (v4l2_fourcc == fmt->repacked_fourcc)
> > + /* Repacking always goes to 16bpp */
> > + return ALIGN(width << 1, BPL_ALIGNMENT);
> > + else
> > + return ALIGN((width * fmt->depth) >> 3, BPL_ALIGNMENT);
> > +}
> > +
> > +static int __subdev_get_format(struct unicam_device *dev,
> > + struct v4l2_mbus_framefmt *fmt, int pad_id)
> > +{
> > + struct v4l2_subdev_format sd_fmt = {
> > + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > + .pad = pad_id
> > + };
> > + int ret;
> > +
> > + ret = v4l2_subdev_call(dev->sensor, pad, get_fmt, dev->sensor_config,
> > + &sd_fmt);
> > + if (ret < 0)
> > + return ret;
> > +
> > + *fmt = sd_fmt.format;
> > +
> > + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__,
> > + fmt->width, fmt->height, fmt->code);
> > +
> > + return 0;
> > +}
> > +
> > +static int __subdev_set_format(struct unicam_device *dev,
> > + struct v4l2_mbus_framefmt *fmt, int pad_id)
> > +{
> > + struct v4l2_subdev_format sd_fmt = {
> > + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > + .pad = pad_id
> > + };
> > + int ret;
> > +
> > + sd_fmt.format = *fmt;
> > +
> > + ret = v4l2_subdev_call(dev->sensor, pad, set_fmt, dev->sensor_config,
> > + &sd_fmt);
> > + if (ret < 0)
> > + return ret;
> > +
> > + *fmt = sd_fmt.format;
> > +
> > + if (pad_id == IMAGE_PAD)
> > + unicam_dbg(1, dev, "%s %dx%d code:%04x\n", __func__, fmt->width,
> > + fmt->height, fmt->code);
> > + else
> > + unicam_dbg(1, dev, "%s Embedded data code:%04x\n", __func__,
> > + sd_fmt.format.code);
> > +
> > + return 0;
> > +}
> > +
> > +static int unicam_calc_format_size_bpl(struct unicam_device *dev,
> > + const struct unicam_fmt *fmt,
> > + struct v4l2_format *f)
> > +{
> > + unsigned int min_bytesperline;
> > +
> > + v4l_bound_align_image(&f->fmt.pix.width, MIN_WIDTH, MAX_WIDTH, 2,
> > + &f->fmt.pix.height, MIN_HEIGHT, MAX_HEIGHT, 0,
> > + 0);
> > +
> > + min_bytesperline = bytes_per_line(f->fmt.pix.width, fmt,
> > + f->fmt.pix.pixelformat);
> > +
> > + if (f->fmt.pix.bytesperline > min_bytesperline &&
> > + f->fmt.pix.bytesperline <= MAX_BYTESPERLINE)
> > + f->fmt.pix.bytesperline = ALIGN(f->fmt.pix.bytesperline,
> > + BPL_ALIGNMENT);
> > + else
> > + f->fmt.pix.bytesperline = min_bytesperline;
> > +
> > + f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
> > +
> > + unicam_dbg(3, dev, "%s: fourcc: %08X size: %dx%d bpl:%d img_size:%d\n",
> > + __func__,
> > + f->fmt.pix.pixelformat,
> > + f->fmt.pix.width, f->fmt.pix.height,
> > + f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
> > +
> > + return 0;
> > +}
> > +
> > +static int unicam_reset_format(struct unicam_node *node)
> > +{
> > + struct unicam_device *dev = node->dev;
> > + struct v4l2_mbus_framefmt mbus_fmt;
> > + int ret;
> > +
> > + if (dev->sensor_embedded_data || node->pad_id != METADATA_PAD) {
> > + ret = __subdev_get_format(dev, &mbus_fmt, node->pad_id);
> > + if (ret) {
> > + unicam_err(dev, "Failed to get_format - ret %d\n", ret);
> > + return ret;
> > + }
> > +
> > + if (mbus_fmt.code != node->fmt->code) {
> > + unicam_err(dev, "code mismatch - fmt->code %08x, mbus_fmt.code %08x\n",
> > + node->fmt->code, mbus_fmt.code);
> > + return ret;
> > + }
> > + }
> > +
> > + if (node->pad_id == IMAGE_PAD) {
> > + v4l2_fill_pix_format(&node->v_fmt.fmt.pix, &mbus_fmt);
> > + node->v_fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > + unicam_calc_format_size_bpl(dev, node->fmt, &node->v_fmt);
> > + } else {
> > + node->v_fmt.type = V4L2_BUF_TYPE_META_CAPTURE;
> > + node->v_fmt.fmt.meta.dataformat = V4L2_META_FMT_SENSOR_DATA;
> > + if (dev->sensor_embedded_data) {
> > + node->v_fmt.fmt.meta.buffersize =
> > + mbus_fmt.width * mbus_fmt.height;
> > + node->embedded_lines = mbus_fmt.height;
> > + } else {
> > + node->v_fmt.fmt.meta.buffersize = UNICAM_EMBEDDED_SIZE;
> > + node->embedded_lines = 1;
> > + }
> > + }
> > +
> > + node->m_fmt = mbus_fmt;
> > + return 0;
> > +}
> > +
> > +static void unicam_wr_dma_addr(struct unicam_device *dev, dma_addr_t dmaaddr,
> > + unsigned int buffer_size, int pad_id)
> > +{
> > + dma_addr_t endaddr = dmaaddr + buffer_size;
> > +
> > + /*
> > + * dmaaddr and endaddr should be a 32-bit address with the top two bits
> > + * set to 0x3 to signify uncached access through the Videocore memory
> > + * controller.
> > + */
> > + WARN_ON((dmaaddr >> 30) != 0x3 || (endaddr >> 30) != 0x3);
> > +
> > + if (pad_id == IMAGE_PAD) {
> > + reg_write(dev, UNICAM_IBSA0, dmaaddr);
> > + reg_write(dev, UNICAM_IBEA0, endaddr);
> > + } else {
> > + reg_write(dev, UNICAM_DBSA0, dmaaddr);
> > + reg_write(dev, UNICAM_DBEA0, endaddr);
> > + }
> > +}
> > +
> > +static inline unsigned int unicam_get_lines_done(struct unicam_device *dev)
> > +{
> > + dma_addr_t start_addr, cur_addr;
> > + unsigned int stride = dev->node[IMAGE_PAD].v_fmt.fmt.pix.bytesperline;
> > + struct unicam_buffer *frm = dev->node[IMAGE_PAD].cur_frm;
> > +
> > + if (!frm)
> > + return 0;
> > +
> > + start_addr = vb2_dma_contig_plane_dma_addr(&frm->vb.vb2_buf, 0);
> > + cur_addr = reg_read(dev, UNICAM_IBWP);
> > + return (unsigned int)(cur_addr - start_addr) / stride;
> > +}
> > +
> > +static inline void unicam_schedule_next_buffer(struct unicam_node *node)
> > +{
> > + struct unicam_device *dev = node->dev;
> > + struct unicam_buffer *buf;
> > + unsigned int size;
> > + dma_addr_t addr;
> > +
> > + buf = list_first_entry(&node->dma_queue, struct unicam_buffer, list);
> > + node->next_frm = buf;
> > + list_del(&buf->list);
> > +
> > + addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
> > + size = (node->pad_id == IMAGE_PAD) ?
> > + node->v_fmt.fmt.pix.sizeimage :
> > + node->v_fmt.fmt.meta.buffersize;
> > +
> > + unicam_wr_dma_addr(dev, addr, size, node->pad_id);
> > +}
> > +
> > +static inline void unicam_schedule_dummy_buffer(struct unicam_node *node)
> > +{
> > + struct unicam_device *dev = node->dev;
> > +
> > + unicam_dbg(3, dev, "Scheduling dummy buffer for node %d\n",
> > + node->pad_id);
> > +
> > + unicam_wr_dma_addr(dev, node->dummy_buf_dma_addr, DUMMY_BUF_SIZE,
> > + node->pad_id);
> > + node->next_frm = NULL;
> > +}
> > +
> > +static inline void unicam_process_buffer_complete(struct unicam_node *node,
> > + unsigned int sequence)
> > +{
> > + node->cur_frm->vb.field = node->m_fmt.field;
> > + node->cur_frm->vb.sequence = sequence;
> > +
> > + vb2_buffer_done(&node->cur_frm->vb.vb2_buf, VB2_BUF_STATE_DONE);
> > +}
> > +
> > +static bool unicam_all_nodes_streaming(struct unicam_device *dev)
> > +{
> > + bool ret;
> > +
> > + ret = dev->node[IMAGE_PAD].open && dev->node[IMAGE_PAD].streaming;
> > + ret &= !dev->node[METADATA_PAD].open ||
> > + dev->node[METADATA_PAD].streaming;
> > + return ret;
> > +}
> > +
> > +static bool unicam_all_nodes_disabled(struct unicam_device *dev)
> > +{
> > + return !dev->node[IMAGE_PAD].streaming &&
> > + !dev->node[METADATA_PAD].streaming;
> > +}
> > +
> > +static void unicam_queue_event_sof(struct unicam_device *unicam)
> > +{
> > + struct v4l2_event event = {
> > + .type = V4L2_EVENT_FRAME_SYNC,
> > + .u.frame_sync.frame_sequence = unicam->sequence,
> > + };
> > +
> > + v4l2_event_queue(&unicam->node[IMAGE_PAD].video_dev, &event);
> > +}
> > +
> > +/*
> > + * unicam_isr : ISR handler for unicam capture
> > + * @irq: irq number
> > + * @dev_id: dev_id ptr
> > + *
> > + * It changes status of the captured buffer, takes next buffer from the queue
> > + * and sets its address in unicam registers
> > + */
> > +static irqreturn_t unicam_isr(int irq, void *dev)
> > +{
> > + struct unicam_device *unicam = dev;
> > + unsigned int lines_done = unicam_get_lines_done(dev);
> > + unsigned int sequence = unicam->sequence;
> > + unsigned int i;
> > + u32 ista, sta;
> > + u64 ts;
> > +
> > + /*
> > + * Don't service interrupts if not streaming.
> > + * Avoids issues if the VPU should enable the
> > + * peripheral without the kernel knowing (that
> > + * shouldn't happen, but causes issues if it does).
> > + */
> > + if (unicam_all_nodes_disabled(unicam))
> > + return IRQ_NONE;
> > +
> > + sta = reg_read(unicam, UNICAM_STA);
> > + /* Write value back to clear the interrupts */
> > + reg_write(unicam, UNICAM_STA, sta);
> > +
> > + ista = reg_read(unicam, UNICAM_ISTA);
> > + /* Write value back to clear the interrupts */
> > + reg_write(unicam, UNICAM_ISTA, ista);
> > +
> > + unicam_dbg(3, unicam, "ISR: ISTA: 0x%X, STA: 0x%X, sequence %d, lines done %d",
> > + ista, sta, sequence, lines_done);
> > +
> > + if (!(sta & (UNICAM_IS | UNICAM_PI0)))
> > + return IRQ_HANDLED;
> > +
> > + /*
> > + * We must run the frame end handler first. If we have a valid next_frm
> > + * and we get a simultaneout FE + FS interrupt, running the FS handler
> > + * first would null out the next_frm ptr and we would have lost the
> > + * buffer forever.
> > + */
> > + if (ista & UNICAM_FEI || sta & UNICAM_PI0) {
> > + /*
> > + * Ensure we have swapped buffers already as we can't
> > + * stop the peripheral. If no buffer is available, use a
> > + * dummy buffer to dump out frames until we get a new buffer
> > + * to use.
> > + */
> > + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > + if (!unicam->node[i].streaming)
> > + continue;
> > +
> > + if (unicam->node[i].cur_frm)
> > + unicam_process_buffer_complete(&unicam->node[i],
> > + sequence);
> > + unicam->node[i].cur_frm = unicam->node[i].next_frm;
> > + }
> > + unicam->sequence++;
> > + }
> > +
> > + if (ista & UNICAM_FSI) {
> > + /*
> > + * Timestamp is to be when the first data byte was captured,
> > + * aka frame start.
> > + */
> > + ts = ktime_get_ns();
> > + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > + if (!unicam->node[i].streaming)
> > + continue;
> > +
> > + if (unicam->node[i].cur_frm)
> > + unicam->node[i].cur_frm->vb.vb2_buf.timestamp =
> > + ts;
> > + /*
> > + * Set the next frame output to go to a dummy frame
> > + * if we have not managed to obtain another frame
> > + * from the queue.
> > + */
> > + unicam_schedule_dummy_buffer(&unicam->node[i]);
> > + }
> > +
> > + unicam_queue_event_sof(unicam);
> > + }
> > +
> > + /*
> > + * Cannot swap buffer at frame end, there may be a race condition
> > + * where the HW does not actually swap it if the new frame has
> > + * already started.
> > + */
> > + if (ista & (UNICAM_FSI | UNICAM_LCI) && !(ista & UNICAM_FEI)) {
> > + for (i = 0; i < ARRAY_SIZE(unicam->node); i++) {
> > + if (!unicam->node[i].streaming)
> > + continue;
> > +
> > + spin_lock(&unicam->node[i].dma_queue_lock);
> > + if (!list_empty(&unicam->node[i].dma_queue) &&
> > + !unicam->node[i].next_frm)
> > + unicam_schedule_next_buffer(&unicam->node[i]);
> > + spin_unlock(&unicam->node[i].dma_queue_lock);
> > + }
> > + }
> > +
> > + if (reg_read(unicam, UNICAM_ICTL) & UNICAM_FCM) {
> > + /* Switch out of trigger mode if selected */
> > + reg_write_field(unicam, UNICAM_ICTL, 1, UNICAM_TFC);
> > + reg_write_field(unicam, UNICAM_ICTL, 0, UNICAM_FCM);
> > + }
> > + return IRQ_HANDLED;
> > +}
> > +
> > +static int unicam_querycap(struct file *file, void *priv,
> > + struct v4l2_capability *cap)
> > +{
> > + struct unicam_node *node = video_drvdata(file);
> > + struct unicam_device *dev = node->dev;
> > +
> > + strlcpy(cap->driver, UNICAM_MODULE_NAME, sizeof(cap->driver));
> > + strlcpy(cap->card, UNICAM_MODULE_NAME, sizeof(cap->card));
>
> strscpy()?
>
> > +
> > + snprintf(cap->bus_info, sizeof(cap->bus_info),
> > + "platform:%s", dev_name(&dev->pdev->dev));
> > +
> > + cap->capabilities |= V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_META_CAPTURE;
>
> Please also fill cap->device_caps.
>
> > +
> > + return 0;
> > +}
> > +
> > +static int unicam_enum_fmt_vid_cap(struct file *file, void *priv,
> > + struct v4l2_fmtdesc *f)
> > +{
> > + struct unicam_node *node = video_drvdata(file);
> > + struct unicam_device *dev = node->dev;
> > + unsigned int index = 0;
> > + unsigned int i;
> > + int ret = 0;
> > +
> > + if (node->pad_id != IMAGE_PAD)
> > + return -EINVAL;
> > +
> > + for (i = 0; !ret && i < MAX_ENUM_MBUS_CODE; i++) {
> > + struct v4l2_subdev_mbus_code_enum mbus_code = {
> > + .index = i,
> > + .pad = IMAGE_PAD,
> > + .which = V4L2_SUBDEV_FORMAT_ACTIVE,
> > + };
> > + const struct unicam_fmt *fmt;
> > +
> > + ret = v4l2_subdev_call(dev->sensor, pad, enum_mbus_code,
> > + NULL, &mbus_code);
> > + if (ret < 0) {
> > + unicam_dbg(2, dev,
> > + "subdev->enum_mbus_code idx %d returned %d - index invalid\n",
> > + i, ret);
> > + return -EINVAL;
> > + }
> > +
> > + fmt = find_format_by_code(mbus_code.code);
> > + if (fmt) {
> > + if (fmt->fourcc) {
> > + if (index == f->index) {
> > + f->pixelformat = fmt->fourcc;
> > + break;
> > + }
> > + index++;
> > + }
> > + if (fmt->repacked_fourcc) {
> > + if (index == f->index) {
> > + f->pixelformat = fmt->repacked_fourcc;
> > + break;
> > + }
> > + index++;
> > + }
> > + }
> > + }
Following on from the above comment, we should check
if (i >= MAX_ENUM_MBUS_CODE)
return -EINVAL;
rather than always returning 0 from here. Possibly add a WARN_ONCE as
well to flag the subdev bug?
Dave
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 05/34] ARM: dts: bcm2711: Add Unicam DT nodes
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (3 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 04/34] media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver Laurent Pinchart
` (29 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
The Unicam IP cores are CSI-2/CCP2 receivers. They are part of VC4, but
can be controlled directly from the operating system without going
through the VPU firmware. Exposes them as DT nodes.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
arch/arm/boot/dts/bcm2711.dtsi | 43 ++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)
diff --git a/arch/arm/boot/dts/bcm2711.dtsi b/arch/arm/boot/dts/bcm2711.dtsi
index a91cf68e3c4c..0a4d75a64e7c 100644
--- a/arch/arm/boot/dts/bcm2711.dtsi
+++ b/arch/arm/boot/dts/bcm2711.dtsi
@@ -2,6 +2,7 @@
#include "bcm283x.dtsi"
#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <dt-bindings/power/raspberrypi-power.h>
#include <dt-bindings/soc/bcm2835-pm.h>
/ {
@@ -244,6 +245,48 @@ pwm1: pwm@7e20c800 {
hvs@7e400000 {
interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>;
};
+
+ csi0: csi@7e800000 {
+ compatible = "brcm,bcm2835-unicam";
+ reg = <0x7e800000 0x800>,
+ <0x7e802000 0x4>;
+ interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clocks BCM2835_CLOCK_CAM0>;
+ clock-names = "lp";
+ power-domains = <&power RPI_POWER_DOMAIN_UNICAM0>;
+ brcm,num-data-lanes = <2>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ #clock-cells = <1>;
+ status = "disabled";
+
+ port {
+ endpoint {
+ data-lanes = <1 2>;
+ };
+ };
+ };
+
+ csi1: csi@7e801000 {
+ compatible = "brcm,bcm2835-unicam";
+ reg = <0x7e801000 0x800>,
+ <0x7e802004 0x4>;
+ interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&clocks BCM2835_CLOCK_CAM1>;
+ clock-names = "lp";
+ power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>;
+ brcm,num-data-lanes = <4>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+ #clock-cells = <1>;
+ status = "disabled";
+
+ port {
+ endpoint {
+ data-lanes = <1 2 3 4>;
+ };
+ };
+ };
};
/*
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (4 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 05/34] ARM: dts: bcm2711: Add Unicam DT nodes Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-05 2:38 ` kbuild test robot
` (4 more replies)
2020-05-04 9:25 ` [PATCH v2 07/34] staging: bcm2835: Break MMAL support out from camera Laurent Pinchart
` (28 subsequent siblings)
34 siblings, 5 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
Add Broadcom VideoCore Shared Memory support.
This new driver allows contiguous memory blocks to be imported
into the VideoCore VPU memory map, and manages the lifetime of
those objects, only releasing the source dmabuf once the VPU has
confirmed it has finished with it.
Driver upported from the RaspberryPi BSP at revision:
890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when counting transactions")
forward ported to recent mainline kernel version.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/Kconfig | 2 +
drivers/staging/vc04_services/Makefile | 1 +
.../include/linux/broadcom/vc_sm_cma_ioctl.h | 114 ++
.../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
.../staging/vc04_services/vc-sm-cma/Makefile | 13 +
drivers/staging/vc04_services/vc-sm-cma/TODO | 1 +
.../staging/vc04_services/vc-sm-cma/vc_sm.c | 1732 +++++++++++++++++
.../staging/vc04_services/vc-sm-cma/vc_sm.h | 84 +
.../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 505 +++++
.../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
.../vc04_services/vc-sm-cma/vc_sm_defs.h | 300 +++
.../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
12 files changed, 2853 insertions(+)
create mode 100644 drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/TODO
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
index 6baf9dd57f1f..b9f1f019ebd8 100644
--- a/drivers/staging/vc04_services/Kconfig
+++ b/drivers/staging/vc04_services/Kconfig
@@ -23,5 +23,7 @@ source "drivers/staging/vc04_services/bcm2835-audio/Kconfig"
source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
+source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
+
endif
diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
index 54d9e2f31916..6e1abf494c1a 100644
--- a/drivers/staging/vc04_services/Makefile
+++ b/drivers/staging/vc04_services/Makefile
@@ -12,6 +12,7 @@ vchiq-objs := \
obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
+obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
ccflags-y += -D__VCCOREVER__=0x04000000
diff --git a/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.h b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.h
new file mode 100644
index 000000000000..107460ad1be3
--- /dev/null
+++ b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.h
@@ -0,0 +1,114 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * Copyright 2019 Raspberry Pi (Trading) Ltd. All rights reserved.
+ *
+ * Based on vmcs_sm_ioctl.h Copyright Broadcom Corporation.
+ */
+
+#ifndef __VC_SM_CMA_IOCTL_H
+#define __VC_SM_CMA_IOCTL_H
+
+/* ---- Include Files ---------------------------------------------------- */
+
+#if defined(__KERNEL__)
+#include <linux/types.h> /* Needed for standard types */
+#else
+#include <stdint.h>
+#endif
+
+#include <linux/ioctl.h>
+
+/* ---- Constants and Types ---------------------------------------------- */
+
+#define VC_SM_CMA_RESOURCE_NAME 32
+#define VC_SM_CMA_RESOURCE_NAME_DEFAULT "sm-host-resource"
+
+/* Type define used to create unique IOCTL number */
+#define VC_SM_CMA_MAGIC_TYPE 'J'
+
+/* IOCTL commands on /dev/vc-sm-cma */
+enum vc_sm_cma_cmd_e {
+ VC_SM_CMA_CMD_ALLOC = 0x5A, /* Start at 0x5A arbitrarily */
+
+ VC_SM_CMA_CMD_IMPORT_DMABUF,
+
+ VC_SM_CMA_CMD_CLEAN_INVALID2,
+
+ VC_SM_CMA_CMD_LAST /* Do not delete */
+};
+
+/* Cache type supported, conveniently matches the user space definition in
+ * user-vcsm.h.
+ */
+enum vc_sm_cma_cache_e {
+ VC_SM_CMA_CACHE_NONE,
+ VC_SM_CMA_CACHE_HOST,
+ VC_SM_CMA_CACHE_VC,
+ VC_SM_CMA_CACHE_BOTH,
+};
+
+/* IOCTL Data structures */
+struct vc_sm_cma_ioctl_alloc {
+ /* user -> kernel */
+ __u32 size;
+ __u32 num;
+ __u32 cached; /* enum vc_sm_cma_cache_e */
+ __u32 pad;
+ __u8 name[VC_SM_CMA_RESOURCE_NAME];
+
+ /* kernel -> user */
+ __s32 handle;
+ __u32 vc_handle;
+ __u64 dma_addr;
+};
+
+struct vc_sm_cma_ioctl_import_dmabuf {
+ /* user -> kernel */
+ __s32 dmabuf_fd;
+ __u32 cached; /* enum vc_sm_cma_cache_e */
+ __u8 name[VC_SM_CMA_RESOURCE_NAME];
+
+ /* kernel -> user */
+ __s32 handle;
+ __u32 vc_handle;
+ __u32 size;
+ __u32 pad;
+ __u64 dma_addr;
+};
+
+/*
+ * Cache functions to be set to struct vc_sm_cma_ioctl_clean_invalid2
+ * invalidate_mode.
+ */
+#define VC_SM_CACHE_OP_NOP 0x00
+#define VC_SM_CACHE_OP_INV 0x01
+#define VC_SM_CACHE_OP_CLEAN 0x02
+#define VC_SM_CACHE_OP_FLUSH 0x03
+
+struct vc_sm_cma_ioctl_clean_invalid2 {
+ __u32 op_count;
+ __u32 pad;
+ struct vc_sm_cma_ioctl_clean_invalid_block {
+ __u32 invalidate_mode;
+ __u32 block_count;
+ void * __user start_address;
+ __u32 block_size;
+ __u32 inter_block_stride;
+ } s[0];
+};
+
+/* IOCTL numbers */
+#define VC_SM_CMA_IOCTL_MEM_ALLOC\
+ _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_ALLOC,\
+ struct vc_sm_cma_ioctl_alloc)
+
+#define VC_SM_CMA_IOCTL_MEM_IMPORT_DMABUF\
+ _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_IMPORT_DMABUF,\
+ struct vc_sm_cma_ioctl_import_dmabuf)
+
+#define VC_SM_CMA_IOCTL_MEM_CLEAN_INVALID2\
+ _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
+ struct vc_sm_cma_ioctl_clean_invalid2)
+
+#endif /* __VC_SM_CMA_IOCTL_H */
diff --git a/drivers/staging/vc04_services/vc-sm-cma/Kconfig b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
new file mode 100644
index 000000000000..5ac115da6b49
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
@@ -0,0 +1,10 @@
+config BCM_VC_SM_CMA
+ bool "VideoCore Shared Memory (CMA) driver"
+ select BCM2835_VCHIQ
+ select RBTREE
+ select DMA_SHARED_BUFFER
+ help
+ Say Y here to enable the shared memory interface that
+ supports sharing dmabufs with VideoCore.
+ This operates over the VCHIQ interface to a service
+ running on VideoCore.
diff --git a/drivers/staging/vc04_services/vc-sm-cma/Makefile b/drivers/staging/vc04_services/vc-sm-cma/Makefile
new file mode 100644
index 000000000000..77d173694fbf
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/Makefile
@@ -0,0 +1,13 @@
+ccflags-y += \
+ -I$(srctree)/$(src)/../ \
+ -I$(srctree)/$(src)/../interface/vchi \
+ -I$(srctree)/$(src)/../interface/vchiq_arm\
+ -I$(srctree)/$(src)/../include
+
+ccflags-y += \
+ -D__VCCOREVER__=0
+
+vc-sm-cma-$(CONFIG_BCM_VC_SM_CMA) := \
+ vc_sm.o vc_sm_cma_vchi.o
+
+obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma.o
diff --git a/drivers/staging/vc04_services/vc-sm-cma/TODO b/drivers/staging/vc04_services/vc-sm-cma/TODO
new file mode 100644
index 000000000000..ac9b5f8a7389
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/TODO
@@ -0,0 +1 @@
+No currently outstanding tasks except some clean-up.
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
new file mode 100644
index 000000000000..cd5fb561debb
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
@@ -0,0 +1,1732 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * VideoCore Shared Memory driver using CMA.
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ * Dave Stevenson <dave.stevenson@raspberrypi.org>
+ *
+ * Based on vmcs_sm driver from Broadcom Corporation for some API,
+ * and taking some code for buffer allocation and dmabuf handling from
+ * videobuf2.
+ *
+ *
+ * This driver has 3 main uses:
+ * 1) Allocating buffers for the kernel or userspace that can be shared with the
+ * VPU.
+ * 2) Importing dmabufs from elsewhere for sharing with the VPU.
+ * 3) Allocating buffers for use by the VPU.
+ *
+ * In the first and second cases the native handle is a dmabuf. Releasing the
+ * resource inherently comes from releasing the dmabuf, and this will trigger
+ * unmapping on the VPU. The underlying allocation and our buffer structure are
+ * retained until the VPU has confirmed that it has finished with it.
+ *
+ * For the VPU allocations the VPU is responsible for triggering the release,
+ * and therefore the released message decrements the dma_buf refcount (with the
+ * VPU mapping having already been marked as released).
+ */
+
+/* ---- Include Files ----------------------------------------------------- */
+#include <linux/cdev.h>
+#include <linux/device.h>
+#include <linux/debugfs.h>
+#include <linux/dma-mapping.h>
+#include <linux/dma-buf.h>
+#include <linux/errno.h>
+#include <linux/fs.h>
+#include <linux/kernel.h>
+#include <linux/list.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+#include <linux/mm.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/proc_fs.h>
+#include <linux/slab.h>
+#include <linux/seq_file.h>
+#include <linux/syscalls.h>
+#include <linux/types.h>
+#include <asm/cacheflush.h>
+
+#include "vchiq_connected.h"
+#include "vc_sm_cma_vchi.h"
+
+#include "vc_sm.h"
+#include "vc_sm_knl.h"
+#include <linux/broadcom/vc_sm_cma_ioctl.h>
+
+/* ---- Private Constants and Types --------------------------------------- */
+
+#define DEVICE_NAME "vcsm-cma"
+#define DEVICE_MINOR 0
+
+#define VC_SM_RESOURCE_NAME_DEFAULT "sm-host-resource"
+
+#define VC_SM_DIR_ROOT_NAME "vcsm-cma"
+#define VC_SM_STATE "state"
+
+/* Private file data associated with each opened device. */
+struct vc_sm_privdata_t {
+ pid_t pid; /* PID of creator. */
+
+ int restart_sys; /* Tracks restart on interrupt. */
+ enum vc_sm_msg_type int_action; /* Interrupted action. */
+ u32 int_trans_id; /* Interrupted transaction. */
+};
+
+typedef int (*VC_SM_SHOW) (struct seq_file *s, void *v);
+struct sm_pde_t {
+ VC_SM_SHOW show; /* Debug fs function hookup. */
+ struct dentry *dir_entry; /* Debug fs directory entry. */
+ void *priv_data; /* Private data */
+};
+
+/* Global state information. */
+struct sm_state_t {
+ struct platform_device *pdev;
+
+ struct miscdevice misc_dev;
+
+ struct sm_instance *sm_handle; /* Handle for videocore service. */
+
+ spinlock_t kernelid_map_lock; /* Spinlock protecting kernelid_map */
+ struct idr kernelid_map;
+
+ struct mutex map_lock; /* Global map lock. */
+ struct list_head buffer_list; /* List of buffer. */
+
+ struct vc_sm_privdata_t *data_knl; /* Kernel internal data tracking. */
+ struct vc_sm_privdata_t *vpu_allocs; /* All allocations from the VPU */
+ struct dentry *dir_root; /* Debug fs entries root. */
+ struct sm_pde_t dir_state; /* Debug fs entries state sub-tree. */
+
+ bool require_released_callback; /* VPU will send a released msg when it
+ * has finished with a resource.
+ */
+ u32 int_trans_id; /* Interrupted transaction. */
+};
+
+struct vc_sm_dma_buf_attachment {
+ struct device *dev;
+ struct sg_table sg_table;
+ struct list_head list;
+ enum dma_data_direction dma_dir;
+};
+
+/* ---- Private Variables ----------------------------------------------- */
+
+static struct sm_state_t *sm_state;
+static int sm_inited;
+
+/* ---- Private Function Prototypes -------------------------------------- */
+
+/* ---- Private Functions ------------------------------------------------ */
+
+static int get_kernel_id(struct vc_sm_buffer *buffer)
+{
+ int handle;
+
+ spin_lock(&sm_state->kernelid_map_lock);
+ handle = idr_alloc(&sm_state->kernelid_map, buffer, 0, 0, GFP_KERNEL);
+ spin_unlock(&sm_state->kernelid_map_lock);
+
+ return handle;
+}
+
+static struct vc_sm_buffer *lookup_kernel_id(int handle)
+{
+ return idr_find(&sm_state->kernelid_map, handle);
+}
+
+static void free_kernel_id(int handle)
+{
+ spin_lock(&sm_state->kernelid_map_lock);
+ idr_remove(&sm_state->kernelid_map, handle);
+ spin_unlock(&sm_state->kernelid_map_lock);
+}
+
+static int vc_sm_cma_seq_file_show(struct seq_file *s, void *v)
+{
+ struct sm_pde_t *sm_pde;
+
+ sm_pde = (struct sm_pde_t *)(s->private);
+
+ if (sm_pde && sm_pde->show)
+ sm_pde->show(s, v);
+
+ return 0;
+}
+
+static int vc_sm_cma_single_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, vc_sm_cma_seq_file_show, inode->i_private);
+}
+
+static const struct file_operations vc_sm_cma_debug_fs_fops = {
+ .open = vc_sm_cma_single_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int vc_sm_cma_global_state_show(struct seq_file *s, void *v)
+{
+ struct vc_sm_buffer *resource = NULL;
+ int resource_count = 0;
+
+ if (!sm_state)
+ return 0;
+
+ seq_printf(s, "\nVC-ServiceHandle %p\n", sm_state->sm_handle);
+
+ /* Log all applicable mapping(s). */
+
+ mutex_lock(&sm_state->map_lock);
+ seq_puts(s, "\nResources\n");
+ if (!list_empty(&sm_state->buffer_list)) {
+ list_for_each_entry(resource, &sm_state->buffer_list,
+ global_buffer_list) {
+ resource_count++;
+
+ seq_printf(s, "\nResource %p\n",
+ resource);
+ seq_printf(s, " NAME %s\n",
+ resource->name);
+ seq_printf(s, " SIZE %zu\n",
+ resource->size);
+ seq_printf(s, " DMABUF %p\n",
+ resource->dma_buf);
+ if (resource->imported) {
+ seq_printf(s, " ATTACH %p\n",
+ resource->import.attach);
+ seq_printf(s, " SGT %p\n",
+ resource->import.sgt);
+ } else {
+ seq_printf(s, " SGT %p\n",
+ resource->alloc.sg_table);
+ }
+ seq_printf(s, " DMA_ADDR %pad\n",
+ &resource->dma_addr);
+ seq_printf(s, " VC_HANDLE %08x\n",
+ resource->vc_handle);
+ seq_printf(s, " VC_MAPPING %d\n",
+ resource->vpu_state);
+ }
+ }
+ seq_printf(s, "\n\nTotal resource count: %d\n\n", resource_count);
+
+ mutex_unlock(&sm_state->map_lock);
+
+ return 0;
+}
+
+/*
+ * Adds a buffer to the private data list which tracks all the allocated
+ * data.
+ */
+static void vc_sm_add_resource(struct vc_sm_privdata_t *privdata,
+ struct vc_sm_buffer *buffer)
+{
+ mutex_lock(&sm_state->map_lock);
+ list_add(&buffer->global_buffer_list, &sm_state->buffer_list);
+ mutex_unlock(&sm_state->map_lock);
+
+ pr_debug("[%s]: added buffer %p (name %s, size %zu)\n",
+ __func__, buffer, buffer->name, buffer->size);
+}
+
+/*
+ * Cleans up imported dmabuf.
+ */
+static void vc_sm_clean_up_dmabuf(struct vc_sm_buffer *buffer)
+{
+ if (!buffer->imported)
+ return;
+
+ /* Handle cleaning up imported dmabufs */
+ mutex_lock(&buffer->lock);
+ if (buffer->import.sgt) {
+ dma_buf_unmap_attachment(buffer->import.attach,
+ buffer->import.sgt,
+ DMA_BIDIRECTIONAL);
+ buffer->import.sgt = NULL;
+ }
+ if (buffer->import.attach) {
+ dma_buf_detach(buffer->dma_buf, buffer->import.attach);
+ buffer->import.attach = NULL;
+ }
+ mutex_unlock(&buffer->lock);
+}
+
+/*
+ * Instructs VPU to decrement the refcount on a buffer.
+ */
+static void vc_sm_vpu_free(struct vc_sm_buffer *buffer)
+{
+ if (buffer->vc_handle && buffer->vpu_state == VPU_MAPPED) {
+ struct vc_sm_free_t free = { buffer->vc_handle, 0 };
+ int status = vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
+ &sm_state->int_trans_id);
+ if (status != 0 && status != -EINTR) {
+ pr_err("[%s]: failed to free memory on videocore (status: %u, trans_id: %u)\n",
+ __func__, status, sm_state->int_trans_id);
+ }
+
+ if (sm_state->require_released_callback) {
+ /* Need to wait for the VPU to confirm the free. */
+
+ /* Retain a reference on this until the VPU has
+ * released it
+ */
+ buffer->vpu_state = VPU_UNMAPPING;
+ } else {
+ buffer->vpu_state = VPU_NOT_MAPPED;
+ buffer->vc_handle = 0;
+ }
+ }
+}
+
+/*
+ * Release an allocation.
+ * All refcounting is done via the dma buf object.
+ *
+ * Must be called with the mutex held. The function will either release the
+ * mutex (if defering the release) or destroy it. The caller must therefore not
+ * reuse the buffer on return.
+ */
+static void vc_sm_release_resource(struct vc_sm_buffer *buffer)
+{
+ pr_debug("[%s]: buffer %p (name %s, size %zu), imported %u\n",
+ __func__, buffer, buffer->name, buffer->size,
+ buffer->imported);
+
+ if (buffer->vc_handle) {
+ /* We've sent the unmap request but not had the response. */
+ pr_debug("[%s]: Waiting for VPU unmap response on %p\n",
+ __func__, buffer);
+ goto defer;
+ }
+ if (buffer->in_use) {
+ /* dmabuf still in use - we await the release */
+ pr_debug("[%s]: buffer %p is still in use\n", __func__, buffer);
+ goto defer;
+ }
+
+ /* Release the allocation (whether imported dmabuf or CMA allocation) */
+ if (buffer->imported) {
+ if (buffer->import.dma_buf)
+ dma_buf_put(buffer->import.dma_buf);
+ else
+ pr_err("%s: Imported dmabuf already been put for buf %p\n",
+ __func__, buffer);
+ buffer->import.dma_buf = NULL;
+ } else {
+ dma_free_coherent(&sm_state->pdev->dev, buffer->size,
+ buffer->cookie, buffer->dma_addr);
+ }
+
+ /* Free our buffer. Start by removing it from the list */
+ mutex_lock(&sm_state->map_lock);
+ list_del(&buffer->global_buffer_list);
+ mutex_unlock(&sm_state->map_lock);
+
+ pr_debug("%s: Release our allocation - done\n", __func__);
+ mutex_unlock(&buffer->lock);
+
+ mutex_destroy(&buffer->lock);
+
+ kfree(buffer);
+ return;
+
+defer:
+ mutex_unlock(&buffer->lock);
+}
+
+/* Create support for private data tracking. */
+static struct vc_sm_privdata_t *vc_sm_cma_create_priv_data(pid_t id)
+{
+ char alloc_name[32];
+ struct vc_sm_privdata_t *file_data = NULL;
+
+ /* Allocate private structure. */
+ file_data = kzalloc(sizeof(*file_data), GFP_KERNEL);
+
+ if (!file_data)
+ return NULL;
+
+ snprintf(alloc_name, sizeof(alloc_name), "%d", id);
+
+ file_data->pid = id;
+
+ return file_data;
+}
+
+/* Dma buf operations for use with our own allocations */
+
+static int vc_sm_dma_buf_attach(struct dma_buf *dmabuf,
+ struct dma_buf_attachment *attachment)
+
+{
+ struct vc_sm_dma_buf_attachment *a;
+ struct sg_table *sgt;
+ struct vc_sm_buffer *buf = dmabuf->priv;
+ struct scatterlist *rd, *wr;
+ int ret, i;
+
+ a = kzalloc(sizeof(*a), GFP_KERNEL);
+ if (!a)
+ return -ENOMEM;
+
+ pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf, attachment);
+
+ mutex_lock(&buf->lock);
+
+ INIT_LIST_HEAD(&a->list);
+
+ sgt = &a->sg_table;
+
+ /* Copy the buf->base_sgt scatter list to the attachment, as we can't
+ * map the same scatter list to multiple attachments at the same time.
+ */
+ ret = sg_alloc_table(sgt, buf->alloc.sg_table->orig_nents, GFP_KERNEL);
+ if (ret) {
+ kfree(a);
+ return -ENOMEM;
+ }
+
+ rd = buf->alloc.sg_table->sgl;
+ wr = sgt->sgl;
+ for (i = 0; i < sgt->orig_nents; ++i) {
+ sg_set_page(wr, sg_page(rd), rd->length, rd->offset);
+ rd = sg_next(rd);
+ wr = sg_next(wr);
+ }
+
+ a->dma_dir = DMA_NONE;
+ attachment->priv = a;
+
+ list_add(&a->list, &buf->attachments);
+ mutex_unlock(&buf->lock);
+
+ return 0;
+}
+
+static void vc_sm_dma_buf_detach(struct dma_buf *dmabuf,
+ struct dma_buf_attachment *attachment)
+{
+ struct vc_sm_dma_buf_attachment *a = attachment->priv;
+ struct vc_sm_buffer *buf = dmabuf->priv;
+ struct sg_table *sgt;
+
+ pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf, attachment);
+ if (!a)
+ return;
+
+ sgt = &a->sg_table;
+
+ /* release the scatterlist cache */
+ if (a->dma_dir != DMA_NONE)
+ dma_unmap_sg(attachment->dev, sgt->sgl, sgt->orig_nents,
+ a->dma_dir);
+ sg_free_table(sgt);
+
+ mutex_lock(&buf->lock);
+ list_del(&a->list);
+ mutex_unlock(&buf->lock);
+
+ kfree(a);
+}
+
+static struct sg_table *vc_sm_map_dma_buf(struct dma_buf_attachment *attachment,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_dma_buf_attachment *a = attachment->priv;
+ /* stealing dmabuf mutex to serialize map/unmap operations */
+ struct mutex *lock = &attachment->dmabuf->lock;
+ struct sg_table *table;
+
+ mutex_lock(lock);
+ pr_debug("%s attachment %p\n", __func__, attachment);
+ table = &a->sg_table;
+
+ /* return previously mapped sg table */
+ if (a->dma_dir == direction) {
+ mutex_unlock(lock);
+ return table;
+ }
+
+ /* release any previous cache */
+ if (a->dma_dir != DMA_NONE) {
+ dma_unmap_sg(attachment->dev, table->sgl, table->orig_nents,
+ a->dma_dir);
+ a->dma_dir = DMA_NONE;
+ }
+
+ /* mapping to the client with new direction */
+ table->nents = dma_map_sg(attachment->dev, table->sgl,
+ table->orig_nents, direction);
+ if (!table->nents) {
+ pr_err("failed to map scatterlist\n");
+ mutex_unlock(lock);
+ return ERR_PTR(-EIO);
+ }
+
+ a->dma_dir = direction;
+ mutex_unlock(lock);
+
+ pr_debug("%s attachment %p\n", __func__, attachment);
+ return table;
+}
+
+static void vc_sm_unmap_dma_buf(struct dma_buf_attachment *attachment,
+ struct sg_table *table,
+ enum dma_data_direction direction)
+{
+ pr_debug("%s attachment %p\n", __func__, attachment);
+ dma_unmap_sg(attachment->dev, table->sgl, table->nents, direction);
+}
+
+static int vc_sm_dmabuf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+ int ret;
+
+ pr_debug("%s dmabuf %p, buf %p, vm_start %08lX\n", __func__, dmabuf,
+ buf, vma->vm_start);
+
+ mutex_lock(&buf->lock);
+
+ /* now map it to userspace */
+ vma->vm_pgoff = 0;
+
+ ret = dma_mmap_coherent(&sm_state->pdev->dev, vma, buf->cookie,
+ buf->dma_addr, buf->size);
+
+ if (ret) {
+ pr_err("Remapping memory failed, error: %d\n", ret);
+ return ret;
+ }
+
+ vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
+
+ mutex_unlock(&buf->lock);
+
+ if (ret)
+ pr_err("%s: failure mapping buffer to userspace\n",
+ __func__);
+
+ return ret;
+}
+
+static void vc_sm_dma_buf_release(struct dma_buf *dmabuf)
+{
+ struct vc_sm_buffer *buffer;
+
+ if (!dmabuf)
+ return;
+
+ buffer = (struct vc_sm_buffer *)dmabuf->priv;
+
+ mutex_lock(&buffer->lock);
+
+ pr_debug("%s dmabuf %p, buffer %p\n", __func__, dmabuf, buffer);
+
+ buffer->in_use = 0;
+
+ /* Unmap on the VPU */
+ vc_sm_vpu_free(buffer);
+ pr_debug("%s vpu_free done\n", __func__);
+
+ /* Unmap our dma_buf object (the vc_sm_buffer remains until released
+ * on the VPU).
+ */
+ vc_sm_clean_up_dmabuf(buffer);
+ pr_debug("%s clean_up dmabuf done\n", __func__);
+
+ /* buffer->lock will be destroyed by vc_sm_release_resource if finished
+ * with, otherwise unlocked. Do NOT unlock here.
+ */
+ vc_sm_release_resource(buffer);
+ pr_debug("%s done\n", __func__);
+}
+
+static int vc_sm_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf;
+ struct vc_sm_dma_buf_attachment *a;
+
+ if (!dmabuf)
+ return -EFAULT;
+
+ buf = dmabuf->priv;
+ if (!buf)
+ return -EFAULT;
+
+ mutex_lock(&buf->lock);
+
+ list_for_each_entry(a, &buf->attachments, list) {
+ dma_sync_sg_for_cpu(a->dev, a->sg_table.sgl,
+ a->sg_table.nents, direction);
+ }
+ mutex_unlock(&buf->lock);
+
+ return 0;
+}
+
+static int vc_sm_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf;
+ struct vc_sm_dma_buf_attachment *a;
+
+ if (!dmabuf)
+ return -EFAULT;
+ buf = dmabuf->priv;
+ if (!buf)
+ return -EFAULT;
+
+ mutex_lock(&buf->lock);
+
+ list_for_each_entry(a, &buf->attachments, list) {
+ dma_sync_sg_for_device(a->dev, a->sg_table.sgl,
+ a->sg_table.nents, direction);
+ }
+ mutex_unlock(&buf->lock);
+
+ return 0;
+}
+
+static const struct dma_buf_ops dma_buf_ops = {
+ .map_dma_buf = vc_sm_map_dma_buf,
+ .unmap_dma_buf = vc_sm_unmap_dma_buf,
+ .mmap = vc_sm_dmabuf_mmap,
+ .release = vc_sm_dma_buf_release,
+ .attach = vc_sm_dma_buf_attach,
+ .detach = vc_sm_dma_buf_detach,
+ .begin_cpu_access = vc_sm_dma_buf_begin_cpu_access,
+ .end_cpu_access = vc_sm_dma_buf_end_cpu_access,
+};
+
+/* Dma_buf operations for chaining through to an imported dma_buf */
+
+static
+int vc_sm_import_dma_buf_attach(struct dma_buf *dmabuf,
+ struct dma_buf_attachment *attachment)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ if (!buf->imported)
+ return -EINVAL;
+ return buf->import.dma_buf->ops->attach(buf->import.dma_buf,
+ attachment);
+}
+
+static
+void vc_sm_import_dma_buf_detatch(struct dma_buf *dmabuf,
+ struct dma_buf_attachment *attachment)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ if (!buf->imported)
+ return;
+ buf->import.dma_buf->ops->detach(buf->import.dma_buf, attachment);
+}
+
+static
+struct sg_table *vc_sm_import_map_dma_buf(struct dma_buf_attachment *attachment,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = attachment->dmabuf->priv;
+
+ if (!buf->imported)
+ return NULL;
+ return buf->import.dma_buf->ops->map_dma_buf(attachment,
+ direction);
+}
+
+static
+void vc_sm_import_unmap_dma_buf(struct dma_buf_attachment *attachment,
+ struct sg_table *table,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = attachment->dmabuf->priv;
+
+ if (!buf->imported)
+ return;
+ buf->import.dma_buf->ops->unmap_dma_buf(attachment, table, direction);
+}
+
+static
+int vc_sm_import_dmabuf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ pr_debug("%s: mmap dma_buf %p, buf %p, imported db %p\n", __func__,
+ dmabuf, buf, buf->import.dma_buf);
+ if (!buf->imported) {
+ pr_err("%s: mmap dma_buf %p- not an imported buffer\n",
+ __func__, dmabuf);
+ return -EINVAL;
+ }
+ return buf->import.dma_buf->ops->mmap(buf->import.dma_buf, vma);
+}
+
+static
+void vc_sm_import_dma_buf_release(struct dma_buf *dmabuf)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ pr_debug("%s: Relasing dma_buf %p\n", __func__, dmabuf);
+ mutex_lock(&buf->lock);
+ if (!buf->imported)
+ return;
+
+ buf->in_use = 0;
+
+ vc_sm_vpu_free(buf);
+
+ vc_sm_release_resource(buf);
+}
+
+static
+int vc_sm_import_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ if (!buf->imported)
+ return -EINVAL;
+ return buf->import.dma_buf->ops->begin_cpu_access(buf->import.dma_buf,
+ direction);
+}
+
+static
+int vc_sm_import_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
+ enum dma_data_direction direction)
+{
+ struct vc_sm_buffer *buf = dmabuf->priv;
+
+ if (!buf->imported)
+ return -EINVAL;
+ return buf->import.dma_buf->ops->end_cpu_access(buf->import.dma_buf,
+ direction);
+}
+
+static const struct dma_buf_ops dma_buf_import_ops = {
+ .map_dma_buf = vc_sm_import_map_dma_buf,
+ .unmap_dma_buf = vc_sm_import_unmap_dma_buf,
+ .mmap = vc_sm_import_dmabuf_mmap,
+ .release = vc_sm_import_dma_buf_release,
+ .attach = vc_sm_import_dma_buf_attach,
+ .detach = vc_sm_import_dma_buf_detatch,
+ .begin_cpu_access = vc_sm_import_dma_buf_begin_cpu_access,
+ .end_cpu_access = vc_sm_import_dma_buf_end_cpu_access,
+};
+
+/* Import a dma_buf to be shared with VC. */
+int
+vc_sm_cma_import_dmabuf_internal(struct vc_sm_privdata_t *private,
+ struct dma_buf *dma_buf,
+ int fd,
+ struct dma_buf **imported_buf)
+{
+ DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct vc_sm_buffer *buffer = NULL;
+ struct vc_sm_import import = { };
+ struct vc_sm_import_result result = { };
+ struct dma_buf_attachment *attach = NULL;
+ struct sg_table *sgt = NULL;
+ dma_addr_t dma_addr;
+ int ret = 0;
+ int status;
+
+ /* Setup our allocation parameters */
+ pr_debug("%s: importing dma_buf %p/fd %d\n", __func__, dma_buf, fd);
+
+ if (fd < 0)
+ get_dma_buf(dma_buf);
+ else
+ dma_buf = dma_buf_get(fd);
+
+ if (!dma_buf)
+ return -EINVAL;
+
+ attach = dma_buf_attach(dma_buf, &sm_state->pdev->dev);
+ if (IS_ERR(attach)) {
+ ret = PTR_ERR(attach);
+ goto error;
+ }
+
+ sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
+ if (IS_ERR(sgt)) {
+ ret = PTR_ERR(sgt);
+ goto error;
+ }
+
+ /* Verify that the address block is contiguous */
+ if (sgt->nents != 1) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ /* Allocate local buffer to track this allocation. */
+ buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
+ if (!buffer) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ import.type = VC_SM_ALLOC_NON_CACHED;
+ dma_addr = sg_dma_address(sgt->sgl);
+ import.addr = (u32)dma_addr;
+ if ((import.addr & 0xC0000000) != 0xC0000000) {
+ pr_err("%s: Expecting an uncached alias for dma_addr %pad\n",
+ __func__, &dma_addr);
+ import.addr |= 0xC0000000;
+ }
+ import.size = sg_dma_len(sgt->sgl);
+ import.allocator = current->tgid;
+ import.kernel_id = get_kernel_id(buffer);
+
+ memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
+ sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
+
+ pr_debug("[%s]: attempt to import \"%s\" data - type %u, addr %pad, size %u.\n",
+ __func__, import.name, import.type, &dma_addr, import.size);
+
+ /* Allocate the videocore buffer. */
+ status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import, &result,
+ &sm_state->int_trans_id);
+ if (status == -EINTR) {
+ pr_debug("[%s]: requesting import memory action restart (trans_id: %u)\n",
+ __func__, sm_state->int_trans_id);
+ ret = -ERESTARTSYS;
+ private->restart_sys = -EINTR;
+ private->int_action = VC_SM_MSG_TYPE_IMPORT;
+ goto error;
+ } else if (status || !result.res_handle) {
+ pr_debug("[%s]: failed to import memory on videocore (status: %u, trans_id: %u)\n",
+ __func__, status, sm_state->int_trans_id);
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ mutex_init(&buffer->lock);
+ INIT_LIST_HEAD(&buffer->attachments);
+ memcpy(buffer->name, import.name,
+ min(sizeof(buffer->name), sizeof(import.name) - 1));
+
+ /* Keep track of the buffer we created. */
+ buffer->private = private;
+ buffer->vc_handle = result.res_handle;
+ buffer->size = import.size;
+ buffer->vpu_state = VPU_MAPPED;
+
+ buffer->imported = 1;
+ buffer->import.dma_buf = dma_buf;
+
+ buffer->import.attach = attach;
+ buffer->import.sgt = sgt;
+ buffer->dma_addr = dma_addr;
+ buffer->in_use = 1;
+ buffer->kernel_id = import.kernel_id;
+
+ /*
+ * We're done - we need to export a new dmabuf chaining through most
+ * functions, but enabling us to release our own internal references
+ * here.
+ */
+ exp_info.ops = &dma_buf_import_ops;
+ exp_info.size = import.size;
+ exp_info.flags = O_RDWR;
+ exp_info.priv = buffer;
+
+ buffer->dma_buf = dma_buf_export(&exp_info);
+ if (IS_ERR(buffer->dma_buf)) {
+ ret = PTR_ERR(buffer->dma_buf);
+ goto error;
+ }
+
+ vc_sm_add_resource(private, buffer);
+
+ *imported_buf = buffer->dma_buf;
+
+ return 0;
+
+error:
+ if (result.res_handle) {
+ struct vc_sm_free_t free = { result.res_handle, 0 };
+
+ vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
+ &sm_state->int_trans_id);
+ }
+ free_kernel_id(import.kernel_id);
+ kfree(buffer);
+ if (sgt)
+ dma_buf_unmap_attachment(attach, sgt, DMA_BIDIRECTIONAL);
+ if (attach)
+ dma_buf_detach(dma_buf, attach);
+ dma_buf_put(dma_buf);
+ return ret;
+}
+
+static int vc_sm_cma_vpu_alloc(u32 size, u32 align, const char *name,
+ u32 mem_handle, struct vc_sm_buffer **ret_buffer)
+{
+ DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct vc_sm_buffer *buffer = NULL;
+ struct sg_table *sgt;
+ int aligned_size;
+ int ret = 0;
+
+ /* Align to the user requested align */
+ aligned_size = ALIGN(size, align);
+ /* and then to a page boundary */
+ aligned_size = PAGE_ALIGN(aligned_size);
+
+ if (!aligned_size)
+ return -EINVAL;
+
+ /* Allocate local buffer to track this allocation. */
+ buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
+ if (!buffer)
+ return -ENOMEM;
+
+ mutex_init(&buffer->lock);
+ /* Acquire the mutex as vc_sm_release_resource will release it in the
+ * error path.
+ */
+ mutex_lock(&buffer->lock);
+
+ buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
+ aligned_size, &buffer->dma_addr,
+ GFP_KERNEL);
+ if (!buffer->cookie) {
+ pr_err("[%s]: dma_alloc_coherent alloc of %d bytes failed\n",
+ __func__, aligned_size);
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ pr_debug("[%s]: alloc of %d bytes success\n",
+ __func__, aligned_size);
+
+ sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
+ if (!sgt) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer->cookie,
+ buffer->dma_addr, buffer->size);
+ if (ret < 0) {
+ pr_err("failed to get scatterlist from DMA API\n");
+ kfree(sgt);
+ ret = -ENOMEM;
+ goto error;
+ }
+ buffer->alloc.sg_table = sgt;
+
+ INIT_LIST_HEAD(&buffer->attachments);
+
+ memcpy(buffer->name, name,
+ min(sizeof(buffer->name), strlen(name)));
+
+ exp_info.ops = &dma_buf_ops;
+ exp_info.size = aligned_size;
+ exp_info.flags = O_RDWR;
+ exp_info.priv = buffer;
+
+ buffer->dma_buf = dma_buf_export(&exp_info);
+ if (IS_ERR(buffer->dma_buf)) {
+ ret = PTR_ERR(buffer->dma_buf);
+ goto error;
+ }
+ buffer->dma_addr = (u32)sg_dma_address(buffer->alloc.sg_table->sgl);
+ if ((buffer->dma_addr & 0xC0000000) != 0xC0000000) {
+ pr_warn_once("%s: Expecting an uncached alias for dma_addr %pad\n",
+ __func__, &buffer->dma_addr);
+ buffer->dma_addr |= 0xC0000000;
+ }
+ buffer->private = sm_state->vpu_allocs;
+
+ buffer->vc_handle = mem_handle;
+ buffer->vpu_state = VPU_MAPPED;
+ buffer->vpu_allocated = 1;
+ buffer->size = size;
+ /*
+ * Create an ID that will be passed along with our message so
+ * that when we service the release reply, we can look up which
+ * resource is being released.
+ */
+ buffer->kernel_id = get_kernel_id(buffer);
+
+ vc_sm_add_resource(sm_state->vpu_allocs, buffer);
+
+ mutex_unlock(&buffer->lock);
+
+ *ret_buffer = buffer;
+ return 0;
+error:
+ if (buffer)
+ vc_sm_release_resource(buffer);
+ return ret;
+}
+
+static void
+vc_sm_vpu_event(struct sm_instance *instance, struct vc_sm_result_t *reply,
+ int reply_len)
+{
+ switch (reply->trans_id & ~0x80000000) {
+ case VC_SM_MSG_TYPE_CLIENT_VERSION:
+ {
+ /* Acknowledge that the firmware supports the version command */
+ pr_debug("%s: firmware acked version msg. Require release cb\n",
+ __func__);
+ sm_state->require_released_callback = true;
+ }
+ break;
+ case VC_SM_MSG_TYPE_RELEASED:
+ {
+ struct vc_sm_released *release = (struct vc_sm_released *)reply;
+ struct vc_sm_buffer *buffer =
+ lookup_kernel_id(release->kernel_id);
+ if (!buffer) {
+ pr_err("%s: VC released a buffer that is already released, kernel_id %d\n",
+ __func__, release->kernel_id);
+ break;
+ }
+ mutex_lock(&buffer->lock);
+
+ pr_debug("%s: Released addr %08x, size %u, id %08x, mem_handle %08x\n",
+ __func__, release->addr, release->size,
+ release->kernel_id, release->vc_handle);
+
+ buffer->vc_handle = 0;
+ buffer->vpu_state = VPU_NOT_MAPPED;
+ free_kernel_id(release->kernel_id);
+
+ if (buffer->vpu_allocated) {
+ /* VPU allocation, so release the dmabuf which will
+ * trigger the clean up.
+ */
+ mutex_unlock(&buffer->lock);
+ dma_buf_put(buffer->dma_buf);
+ } else {
+ vc_sm_release_resource(buffer);
+ }
+ }
+ break;
+ case VC_SM_MSG_TYPE_VC_MEM_REQUEST:
+ {
+ struct vc_sm_buffer *buffer = NULL;
+ struct vc_sm_vc_mem_request *req =
+ (struct vc_sm_vc_mem_request *)reply;
+ struct vc_sm_vc_mem_request_result reply;
+ int ret;
+
+ pr_debug("%s: Request %u bytes of memory, align %d name %s, trans_id %08x\n",
+ __func__, req->size, req->align, req->name,
+ req->trans_id);
+ ret = vc_sm_cma_vpu_alloc(req->size, req->align, req->name,
+ req->vc_handle, &buffer);
+
+ reply.trans_id = req->trans_id;
+ if (!ret) {
+ reply.addr = buffer->dma_addr;
+ reply.kernel_id = buffer->kernel_id;
+ pr_debug("%s: Allocated resource buffer %p, addr %pad\n",
+ __func__, buffer, &buffer->dma_addr);
+ } else {
+ pr_err("%s: Allocation failed size %u, name %s, vc_handle %u\n",
+ __func__, req->size, req->name, req->vc_handle);
+ reply.addr = 0;
+ reply.kernel_id = 0;
+ }
+ vc_sm_vchi_client_vc_mem_req_reply(sm_state->sm_handle, &reply,
+ &sm_state->int_trans_id);
+ break;
+ }
+ break;
+ default:
+ pr_err("%s: Unknown vpu cmd %x\n", __func__, reply->trans_id);
+ break;
+ }
+}
+
+/* Userspace handling */
+/*
+ * Open the device. Creates a private state to help track all allocation
+ * associated with this device.
+ */
+static int vc_sm_cma_open(struct inode *inode, struct file *file)
+{
+ /* Make sure the device was started properly. */
+ if (!sm_state) {
+ pr_err("[%s]: invalid device\n", __func__);
+ return -EPERM;
+ }
+
+ file->private_data = vc_sm_cma_create_priv_data(current->tgid);
+ if (!file->private_data) {
+ pr_err("[%s]: failed to create data tracker\n", __func__);
+
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+/*
+ * Close the vcsm-cma device.
+ * All allocations are file descriptors to the dmabuf objects, so we will get
+ * the clean up request on those as those are cleaned up.
+ */
+static int vc_sm_cma_release(struct inode *inode, struct file *file)
+{
+ struct vc_sm_privdata_t *file_data =
+ (struct vc_sm_privdata_t *)file->private_data;
+ int ret = 0;
+
+ /* Make sure the device was started properly. */
+ if (!sm_state || !file_data) {
+ pr_err("[%s]: invalid device\n", __func__);
+ ret = -EPERM;
+ goto out;
+ }
+
+ pr_debug("[%s]: using private data %p\n", __func__, file_data);
+
+ /* Terminate the private data. */
+ kfree(file_data);
+
+out:
+ return ret;
+}
+
+/*
+ * Allocate a shared memory handle and block.
+ * Allocation is from CMA, and then imported into the VPU mappings.
+ */
+int vc_sm_cma_ioctl_alloc(struct vc_sm_privdata_t *private,
+ struct vc_sm_cma_ioctl_alloc *ioparam)
+{
+ DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct vc_sm_buffer *buffer = NULL;
+ struct vc_sm_import import = { 0 };
+ struct vc_sm_import_result result = { 0 };
+ struct dma_buf *dmabuf = NULL;
+ struct sg_table *sgt;
+ int aligned_size;
+ int ret = 0;
+ int status;
+ int fd = -1;
+
+ aligned_size = PAGE_ALIGN(ioparam->size);
+
+ if (!aligned_size)
+ return -EINVAL;
+
+ /* Allocate local buffer to track this allocation. */
+ buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
+ if (!buffer) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
+ aligned_size,
+ &buffer->dma_addr,
+ GFP_KERNEL);
+ if (!buffer->cookie) {
+ pr_err("[%s]: dma_alloc_coherent alloc of %d bytes failed\n",
+ __func__, aligned_size);
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ import.type = VC_SM_ALLOC_NON_CACHED;
+ import.allocator = current->tgid;
+
+ if (*ioparam->name)
+ memcpy(import.name, ioparam->name, sizeof(import.name) - 1);
+ else
+ memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
+ sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
+
+ mutex_init(&buffer->lock);
+ INIT_LIST_HEAD(&buffer->attachments);
+ memcpy(buffer->name, import.name,
+ min(sizeof(buffer->name), sizeof(import.name) - 1));
+
+ exp_info.ops = &dma_buf_ops;
+ exp_info.size = aligned_size;
+ exp_info.flags = O_RDWR;
+ exp_info.priv = buffer;
+
+ dmabuf = dma_buf_export(&exp_info);
+ if (IS_ERR(dmabuf)) {
+ ret = PTR_ERR(dmabuf);
+ goto error;
+ }
+ buffer->dma_buf = dmabuf;
+
+ import.addr = buffer->dma_addr;
+ import.size = aligned_size;
+ import.kernel_id = get_kernel_id(buffer);
+
+ /* Wrap it into a videocore buffer. */
+ status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import, &result,
+ &sm_state->int_trans_id);
+ if (status == -EINTR) {
+ pr_debug("[%s]: requesting import memory action restart (trans_id: %u)\n",
+ __func__, sm_state->int_trans_id);
+ ret = -ERESTARTSYS;
+ private->restart_sys = -EINTR;
+ private->int_action = VC_SM_MSG_TYPE_IMPORT;
+ goto error;
+ } else if (status || !result.res_handle) {
+ pr_err("[%s]: failed to import memory on videocore (status: %u, trans_id: %u)\n",
+ __func__, status, sm_state->int_trans_id);
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ /* Keep track of the buffer we created. */
+ buffer->private = private;
+ buffer->vc_handle = result.res_handle;
+ buffer->size = import.size;
+ buffer->vpu_state = VPU_MAPPED;
+ buffer->kernel_id = import.kernel_id;
+
+ sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
+ if (!sgt) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer->cookie,
+ buffer->dma_addr, buffer->size);
+ if (ret < 0) {
+ /* FIXME: error handling */
+ pr_err("failed to get scatterlist from DMA API\n");
+ kfree(sgt);
+ ret = -ENOMEM;
+ goto error;
+ }
+ buffer->alloc.sg_table = sgt;
+
+ fd = dma_buf_fd(dmabuf, O_CLOEXEC);
+ if (fd < 0)
+ goto error;
+
+ vc_sm_add_resource(private, buffer);
+
+ pr_debug("[%s]: Added resource as fd %d, buffer %p, private %p, dma_addr %pad\n",
+ __func__, fd, buffer, private, &buffer->dma_addr);
+
+ /* We're done */
+ ioparam->handle = fd;
+ ioparam->vc_handle = buffer->vc_handle;
+ ioparam->dma_addr = buffer->dma_addr;
+ return 0;
+
+error:
+ pr_err("[%s]: something failed - cleanup. ret %d\n", __func__, ret);
+
+ if (dmabuf) {
+ /* dmabuf has been exported, therefore allow dmabuf cleanup to
+ * deal with this
+ */
+ dma_buf_put(dmabuf);
+ } else {
+ /* No dmabuf, therefore just free the buffer here */
+ if (buffer->cookie)
+ dma_free_coherent(&sm_state->pdev->dev, buffer->size,
+ buffer->cookie, buffer->dma_addr);
+ kfree(buffer);
+ }
+ return ret;
+}
+
+#ifndef CONFIG_ARM64
+/* Converts VCSM_CACHE_OP_* to an operating function. */
+static void (*cache_op_to_func(const unsigned int cache_op))
+ (const void*, const void*)
+{
+ switch (cache_op) {
+ case VC_SM_CACHE_OP_NOP:
+ return NULL;
+
+ case VC_SM_CACHE_OP_INV:
+ case VC_SM_CACHE_OP_CLEAN:
+ case VC_SM_CACHE_OP_FLUSH:
+ return dmac_flush_range;
+
+ default:
+ pr_err("[%s]: Invalid cache_op: 0x%08x\n", __func__, cache_op);
+ return NULL;
+ }
+}
+
+/*
+ * Clean/invalid/flush cache of which buffer is already pinned (i.e. accessed).
+ */
+static int clean_invalid_contig_2d(const void __user *addr,
+ const size_t block_count,
+ const size_t block_size,
+ const size_t stride,
+ const unsigned int cache_op)
+{
+ size_t i;
+ void (*op_fn)(const void *start, const void *end);
+
+ if (!block_size) {
+ pr_err("[%s]: size cannot be 0\n", __func__);
+ return -EINVAL;
+ }
+
+ op_fn = cache_op_to_func(cache_op);
+ if (!op_fn)
+ return -EINVAL;
+
+ for (i = 0; i < block_count; i ++, addr += stride)
+ op_fn(addr, addr + block_size);
+
+ return 0;
+}
+
+static int vc_sm_cma_clean_invalid2(unsigned int cmdnr, unsigned long arg)
+{
+ struct vc_sm_cma_ioctl_clean_invalid2 ioparam;
+ struct vc_sm_cma_ioctl_clean_invalid_block *block = NULL;
+ int i, ret = 0;
+
+ /* Get parameter data. */
+ if (copy_from_user(&ioparam, (void *)arg, sizeof(ioparam))) {
+ pr_err("[%s]: failed to copy-from-user header for cmd %x\n",
+ __func__, cmdnr);
+ return -EFAULT;
+ }
+ block = kmalloc(ioparam.op_count * sizeof(*block), GFP_KERNEL);
+ if (!block)
+ return -EFAULT;
+
+ if (copy_from_user(block, (void *)(arg + sizeof(ioparam)),
+ ioparam.op_count * sizeof(*block)) != 0) {
+ pr_err("[%s]: failed to copy-from-user payload for cmd %x\n",
+ __func__, cmdnr);
+ ret = -EFAULT;
+ goto out;
+ }
+
+ for (i = 0; i < ioparam.op_count; i++) {
+ const struct vc_sm_cma_ioctl_clean_invalid_block * const op =
+ block + i;
+
+ if (op->invalidate_mode == VC_SM_CACHE_OP_NOP)
+ continue;
+
+ ret = clean_invalid_contig_2d((void __user *)op->start_address,
+ op->block_count, op->block_size,
+ op->inter_block_stride,
+ op->invalidate_mode);
+ if (ret)
+ break;
+ }
+out:
+ kfree(block);
+
+ return ret;
+}
+#endif
+
+static long vc_sm_cma_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ int ret = 0;
+ unsigned int cmdnr = _IOC_NR(cmd);
+ struct vc_sm_privdata_t *file_data =
+ (struct vc_sm_privdata_t *)file->private_data;
+
+ /* Validate we can work with this device. */
+ if (!sm_state || !file_data) {
+ pr_err("[%s]: invalid device\n", __func__);
+ return -EPERM;
+ }
+
+ /* Action is a re-post of a previously interrupted action? */
+ if (file_data->restart_sys == -EINTR) {
+ struct vc_sm_action_clean_t action_clean;
+
+ pr_debug("[%s]: clean up of action %u (trans_id: %u) following EINTR\n",
+ __func__, file_data->int_action,
+ file_data->int_trans_id);
+
+ action_clean.res_action = file_data->int_action;
+ action_clean.action_trans_id = file_data->int_trans_id;
+
+ file_data->restart_sys = 0;
+ }
+
+ /* Now process the command. */
+ switch (cmdnr) {
+ /* New memory allocation.
+ */
+ case VC_SM_CMA_CMD_ALLOC:
+ {
+ struct vc_sm_cma_ioctl_alloc ioparam;
+
+ /* Get the parameter data. */
+ if (copy_from_user
+ (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
+ pr_err("[%s]: failed to copy-from-user for cmd %x\n",
+ __func__, cmdnr);
+ ret = -EFAULT;
+ break;
+ }
+
+ ret = vc_sm_cma_ioctl_alloc(file_data, &ioparam);
+ if (!ret &&
+ (copy_to_user((void *)arg, &ioparam,
+ sizeof(ioparam)) != 0)) {
+ /* FIXME: Release allocation */
+ pr_err("[%s]: failed to copy-to-user for cmd %x\n",
+ __func__, cmdnr);
+ ret = -EFAULT;
+ }
+ break;
+ }
+
+ case VC_SM_CMA_CMD_IMPORT_DMABUF:
+ {
+ struct vc_sm_cma_ioctl_import_dmabuf ioparam;
+ struct dma_buf *new_dmabuf;
+
+ /* Get the parameter data. */
+ if (copy_from_user
+ (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
+ pr_err("[%s]: failed to copy-from-user for cmd %x\n",
+ __func__, cmdnr);
+ ret = -EFAULT;
+ break;
+ }
+
+ ret = vc_sm_cma_import_dmabuf_internal(file_data,
+ NULL,
+ ioparam.dmabuf_fd,
+ &new_dmabuf);
+
+ if (!ret) {
+ struct vc_sm_buffer *buf = new_dmabuf->priv;
+
+ ioparam.size = buf->size;
+ ioparam.handle = dma_buf_fd(new_dmabuf,
+ O_CLOEXEC);
+ ioparam.vc_handle = buf->vc_handle;
+ ioparam.dma_addr = buf->dma_addr;
+
+ if (ioparam.handle < 0 ||
+ (copy_to_user((void *)arg, &ioparam,
+ sizeof(ioparam)) != 0)) {
+ dma_buf_put(new_dmabuf);
+ /* FIXME: Release allocation */
+ ret = -EFAULT;
+ }
+ }
+ break;
+ }
+
+#ifndef CONFIG_ARM64
+ /*
+ * Flush/Invalidate the cache for a given mapping.
+ * Blocks must be pinned (i.e. accessed) before this call.
+ */
+ case VC_SM_CMA_CMD_CLEAN_INVALID2:
+ ret = vc_sm_cma_clean_invalid2(cmdnr, arg);
+ break;
+#endif
+
+ default:
+ pr_debug("[%s]: cmd %x tgid %u, owner %u\n", __func__, cmdnr,
+ current->tgid, file_data->pid);
+
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+#ifdef CONFIG_COMPAT
+struct vc_sm_cma_ioctl_clean_invalid2_32 {
+ u32 op_count;
+ struct vc_sm_cma_ioctl_clean_invalid_block_32 {
+ u16 invalidate_mode;
+ u16 block_count;
+ compat_uptr_t start_address;
+ u32 block_size;
+ u32 inter_block_stride;
+ } s[0];
+};
+
+#define VC_SM_CMA_CMD_CLEAN_INVALID2_32\
+ _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
+ struct vc_sm_cma_ioctl_clean_invalid2_32)
+
+static long vc_sm_cma_compat_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ switch (cmd) {
+ case VC_SM_CMA_CMD_CLEAN_INVALID2_32:
+ /* FIXME */
+ return -EINVAL;
+
+ default:
+ return vc_sm_cma_ioctl(file, cmd, arg);
+ }
+}
+#endif
+
+/* Device operations that we managed in this driver. */
+static const struct file_operations vc_sm_ops = {
+ .owner = THIS_MODULE,
+ .unlocked_ioctl = vc_sm_cma_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = vc_sm_cma_compat_ioctl,
+#endif
+ .open = vc_sm_cma_open,
+ .release = vc_sm_cma_release,
+};
+
+/* Driver load/unload functions */
+/* Videocore connected. */
+static void vc_sm_connected_init(void)
+{
+ int ret;
+ struct vchi_instance_handle *vchi_instance;
+ struct vc_sm_version version;
+ struct vc_sm_result_t version_result;
+
+ pr_info("[%s]: start\n", __func__);
+
+ /*
+ * Initialize and create a VCHI connection for the shared memory service
+ * running on videocore.
+ */
+ ret = vchi_initialise(&vchi_instance);
+ if (ret) {
+ pr_err("[%s]: failed to initialise VCHI instance (ret=%d)\n",
+ __func__, ret);
+
+ return;
+ }
+
+ ret = vchi_connect(vchi_instance);
+ if (ret) {
+ pr_err("[%s]: failed to connect VCHI instance (ret=%d)\n",
+ __func__, ret);
+
+ return;
+ }
+
+ /* Initialize an instance of the shared memory service. */
+ sm_state->sm_handle = vc_sm_cma_vchi_init(vchi_instance, 1,
+ vc_sm_vpu_event);
+ if (!sm_state->sm_handle) {
+ pr_err("[%s]: failed to initialize shared memory service\n",
+ __func__);
+
+ return;
+ }
+
+ /* Create a debug fs directory entry (root). */
+ sm_state->dir_root = debugfs_create_dir(VC_SM_DIR_ROOT_NAME, NULL);
+
+ sm_state->dir_state.show = &vc_sm_cma_global_state_show;
+ sm_state->dir_state.dir_entry =
+ debugfs_create_file(VC_SM_STATE, 0444, sm_state->dir_root,
+ &sm_state->dir_state,
+ &vc_sm_cma_debug_fs_fops);
+
+ INIT_LIST_HEAD(&sm_state->buffer_list);
+
+ /* Create a shared memory device. */
+ sm_state->misc_dev.minor = MISC_DYNAMIC_MINOR;
+ sm_state->misc_dev.name = DEVICE_NAME;
+ sm_state->misc_dev.fops = &vc_sm_ops;
+ sm_state->misc_dev.parent = NULL;
+ /* Temporarily set as 666 until udev rules have been sorted */
+ sm_state->misc_dev.mode = 0666;
+ ret = misc_register(&sm_state->misc_dev);
+ if (ret) {
+ pr_err("vcsm-cma: failed to register misc device.\n");
+ goto err_remove_debugfs;
+ }
+
+ sm_state->data_knl = vc_sm_cma_create_priv_data(0);
+ if (!sm_state->data_knl) {
+ pr_err("[%s]: failed to create kernel private data tracker\n",
+ __func__);
+ goto err_remove_misc_dev;
+ }
+
+ version.version = 2;
+ ret = vc_sm_cma_vchi_client_version(sm_state->sm_handle, &version,
+ &version_result,
+ &sm_state->int_trans_id);
+ if (ret) {
+ pr_err("[%s]: Failed to send version request %d\n", __func__,
+ ret);
+ }
+
+ /* Done! */
+ sm_inited = 1;
+ pr_info("[%s]: installed successfully\n", __func__);
+ return;
+
+err_remove_misc_dev:
+ misc_deregister(&sm_state->misc_dev);
+err_remove_debugfs:
+ debugfs_remove_recursive(sm_state->dir_root);
+ vc_sm_cma_vchi_stop(&sm_state->sm_handle);
+}
+
+/* Driver loading. */
+static int bcm2835_vc_sm_cma_probe(struct platform_device *pdev)
+{
+ pr_info("%s: Videocore shared memory driver\n", __func__);
+
+ sm_state = devm_kzalloc(&pdev->dev, sizeof(*sm_state), GFP_KERNEL);
+ if (!sm_state)
+ return -ENOMEM;
+ sm_state->pdev = pdev;
+ mutex_init(&sm_state->map_lock);
+
+ spin_lock_init(&sm_state->kernelid_map_lock);
+ idr_init_base(&sm_state->kernelid_map, 1);
+
+ pdev->dev.dma_parms = devm_kzalloc(&pdev->dev,
+ sizeof(*pdev->dev.dma_parms),
+ GFP_KERNEL);
+ /* dma_set_max_seg_size checks if dma_parms is NULL. */
+ dma_set_max_seg_size(&pdev->dev, 0x3FFFFFFF);
+
+ vchiq_add_connected_callback(vc_sm_connected_init);
+ return 0;
+}
+
+/* Driver unloading. */
+static int bcm2835_vc_sm_cma_remove(struct platform_device *pdev)
+{
+ pr_debug("[%s]: start\n", __func__);
+ if (sm_inited) {
+ misc_deregister(&sm_state->misc_dev);
+
+ /* Remove all proc entries. */
+ debugfs_remove_recursive(sm_state->dir_root);
+
+ /* Stop the videocore shared memory service. */
+ vc_sm_cma_vchi_stop(&sm_state->sm_handle);
+ }
+
+ if (sm_state) {
+ idr_destroy(&sm_state->kernelid_map);
+
+ /* Free the memory for the state structure. */
+ mutex_destroy(&sm_state->map_lock);
+ }
+
+ pr_debug("[%s]: end\n", __func__);
+ return 0;
+}
+
+/* Kernel API calls */
+/* Get an internal resource handle mapped from the external one. */
+int vc_sm_cma_int_handle(void *handle)
+{
+ struct dma_buf *dma_buf = (struct dma_buf *)handle;
+ struct vc_sm_buffer *buf;
+
+ /* Validate we can work with this device. */
+ if (!sm_state || !handle) {
+ pr_err("[%s]: invalid input\n", __func__);
+ return 0;
+ }
+
+ buf = (struct vc_sm_buffer *)dma_buf->priv;
+ return buf->vc_handle;
+}
+EXPORT_SYMBOL_GPL(vc_sm_cma_int_handle);
+
+/* Free a previously allocated shared memory handle and block. */
+int vc_sm_cma_free(void *handle)
+{
+ struct dma_buf *dma_buf = (struct dma_buf *)handle;
+
+ /* Validate we can work with this device. */
+ if (!sm_state || !handle) {
+ pr_err("[%s]: invalid input\n", __func__);
+ return -EPERM;
+ }
+
+ pr_debug("%s: handle %p/dmabuf %p\n", __func__, handle, dma_buf);
+
+ dma_buf_put(dma_buf);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vc_sm_cma_free);
+
+/* Import a dmabuf to be shared with VC. */
+int vc_sm_cma_import_dmabuf(struct dma_buf *src_dmabuf, void **handle)
+{
+ struct dma_buf *new_dma_buf;
+ struct vc_sm_buffer *buf;
+ int ret;
+
+ /* Validate we can work with this device. */
+ if (!sm_state || !src_dmabuf || !handle) {
+ pr_err("[%s]: invalid input\n", __func__);
+ return -EPERM;
+ }
+
+ ret = vc_sm_cma_import_dmabuf_internal(sm_state->data_knl, src_dmabuf,
+ -1, &new_dma_buf);
+
+ if (!ret) {
+ pr_debug("%s: imported to ptr %p\n", __func__, new_dma_buf);
+ buf = (struct vc_sm_buffer *)new_dma_buf->priv;
+
+ /* Assign valid handle at this time.*/
+ *handle = new_dma_buf;
+ } else {
+ /*
+ * succeeded in importing the dma_buf, but then
+ * failed to look it up again. How?
+ * Release the fd again.
+ */
+ pr_err("%s: imported vc_sm_cma_get_buffer failed %d\n",
+ __func__, ret);
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(vc_sm_cma_import_dmabuf);
+
+static struct platform_driver bcm2835_vcsm_cma_driver = {
+ .probe = bcm2835_vc_sm_cma_probe,
+ .remove = bcm2835_vc_sm_cma_remove,
+ .driver = {
+ .name = DEVICE_NAME,
+ .owner = THIS_MODULE,
+ },
+};
+
+module_platform_driver(bcm2835_vcsm_cma_driver);
+
+MODULE_AUTHOR("Dave Stevenson");
+MODULE_DESCRIPTION("VideoCore CMA Shared Memory Driver");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS("platform:vcsm-cma");
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
new file mode 100644
index 000000000000..f1c7b95b14ce
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
@@ -0,0 +1,84 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory driver using CMA.
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ *
+ */
+
+#ifndef VC_SM_H
+#define VC_SM_H
+
+#include <linux/device.h>
+#include <linux/dma-direction.h>
+#include <linux/kref.h>
+#include <linux/mm_types.h>
+#include <linux/mutex.h>
+#include <linux/rbtree.h>
+#include <linux/sched.h>
+#include <linux/shrinker.h>
+#include <linux/types.h>
+#include <linux/miscdevice.h>
+
+#define VC_SM_MAX_NAME_LEN 32
+
+enum vc_sm_vpu_mapping_state {
+ VPU_NOT_MAPPED,
+ VPU_MAPPED,
+ VPU_UNMAPPING
+};
+
+struct vc_sm_alloc_data {
+ unsigned long num_pages;
+ void *priv_virt;
+ struct sg_table *sg_table;
+};
+
+struct vc_sm_imported {
+ struct dma_buf *dma_buf;
+ struct dma_buf_attachment *attach;
+ struct sg_table *sgt;
+};
+
+struct vc_sm_buffer {
+ struct list_head global_buffer_list; /* Global list of buffers. */
+
+ /* Index in the kernel_id idr so that we can find the
+ * mmal_msg_context again when servicing the VCHI reply.
+ */
+ int kernel_id;
+
+ size_t size;
+
+ /* Lock over all the following state for this buffer */
+ struct mutex lock;
+ struct list_head attachments;
+
+ char name[VC_SM_MAX_NAME_LEN];
+
+ int in_use:1; /* Kernel is still using this resource */
+ int imported:1; /* Imported dmabuf */
+
+ enum vc_sm_vpu_mapping_state vpu_state;
+ u32 vc_handle; /* VideoCore handle for this buffer */
+ int vpu_allocated; /*
+ * The VPU made this allocation. Release the
+ * local dma_buf when the VPU releases the
+ * resource.
+ */
+
+ /* DMABUF related fields */
+ struct dma_buf *dma_buf;
+ dma_addr_t dma_addr;
+ void *cookie;
+
+ struct vc_sm_privdata_t *private;
+
+ union {
+ struct vc_sm_alloc_data alloc;
+ struct vc_sm_imported import;
+ };
+};
+
+#endif
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
new file mode 100644
index 000000000000..6a203c60bf7f
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
@@ -0,0 +1,505 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
+ *
+ * Based on vmcs_sm driver from Broadcom Corporation.
+ *
+ */
+
+/* ---- Include Files ----------------------------------------------------- */
+#include <linux/completion.h>
+#include <linux/kernel.h>
+#include <linux/kthread.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
+#include <linux/semaphore.h>
+#include <linux/slab.h>
+#include <linux/types.h>
+
+#include "vc_sm_cma_vchi.h"
+
+#define VC_SM_VER 1
+#define VC_SM_MIN_VER 0
+
+/* ---- Private Constants and Types -------------------------------------- */
+
+/* Command blocks come from a pool */
+#define SM_MAX_NUM_CMD_RSP_BLKS 32
+
+struct sm_cmd_rsp_blk {
+ struct list_head head; /* To create lists */
+ /* To be signaled when the response is there */
+ struct completion cmplt;
+
+ u32 id;
+ u16 length;
+
+ u8 msg[VC_SM_MAX_MSG_LEN];
+
+ uint32_t wait:1;
+ uint32_t sent:1;
+ uint32_t alloc:1;
+
+};
+
+struct sm_instance {
+ u32 num_connections;
+ struct vchi_service_handle *vchi_handle[VCHI_MAX_NUM_CONNECTIONS];
+ struct task_struct *io_thread;
+ struct completion io_cmplt;
+
+ vpu_event_cb vpu_event;
+
+ /* Mutex over the following lists */
+ struct mutex lock;
+ u32 trans_id;
+ struct list_head cmd_list;
+ struct list_head rsp_list;
+ struct list_head dead_list;
+
+ struct sm_cmd_rsp_blk free_blk[SM_MAX_NUM_CMD_RSP_BLKS];
+
+ /* Mutex over the free_list */
+ struct mutex free_lock;
+ struct list_head free_list;
+
+ struct semaphore free_sema;
+
+};
+
+/* ---- Private Variables ------------------------------------------------ */
+
+/* ---- Private Function Prototypes -------------------------------------- */
+
+/* ---- Private Functions ------------------------------------------------ */
+static int
+bcm2835_vchi_msg_queue(struct vchi_service_handle *handle,
+ void *data,
+ unsigned int size)
+{
+ return vchi_queue_kernel_message(handle,
+ data,
+ size);
+}
+
+static struct
+sm_cmd_rsp_blk *vc_vchi_cmd_create(struct sm_instance *instance,
+ enum vc_sm_msg_type id, void *msg,
+ u32 size, int wait)
+{
+ struct sm_cmd_rsp_blk *blk;
+ struct vc_sm_msg_hdr_t *hdr;
+
+ if (down_interruptible(&instance->free_sema)) {
+ blk = kmalloc(sizeof(*blk), GFP_KERNEL);
+ if (!blk)
+ return NULL;
+
+ blk->alloc = 1;
+ init_completion(&blk->cmplt);
+ } else {
+ mutex_lock(&instance->free_lock);
+ blk =
+ list_first_entry(&instance->free_list,
+ struct sm_cmd_rsp_blk, head);
+ list_del(&blk->head);
+ mutex_unlock(&instance->free_lock);
+ }
+
+ blk->sent = 0;
+ blk->wait = wait;
+ blk->length = sizeof(*hdr) + size;
+
+ hdr = (struct vc_sm_msg_hdr_t *)blk->msg;
+ hdr->type = id;
+ mutex_lock(&instance->lock);
+ instance->trans_id++;
+ /*
+ * Retain the top bit for identifying asynchronous events, or VPU cmds.
+ */
+ instance->trans_id &= ~0x80000000;
+ hdr->trans_id = instance->trans_id;
+ blk->id = instance->trans_id;
+ mutex_unlock(&instance->lock);
+
+ if (size)
+ memcpy(hdr->body, msg, size);
+
+ return blk;
+}
+
+static void
+vc_vchi_cmd_delete(struct sm_instance *instance, struct sm_cmd_rsp_blk *blk)
+{
+ if (blk->alloc) {
+ kfree(blk);
+ return;
+ }
+
+ mutex_lock(&instance->free_lock);
+ list_add(&blk->head, &instance->free_list);
+ mutex_unlock(&instance->free_lock);
+ up(&instance->free_sema);
+}
+
+static void vc_sm_cma_vchi_rx_ack(struct sm_instance *instance,
+ struct sm_cmd_rsp_blk *cmd,
+ struct vc_sm_result_t *reply,
+ u32 reply_len)
+{
+ mutex_lock(&instance->lock);
+ list_for_each_entry(cmd,
+ &instance->rsp_list,
+ head) {
+ if (cmd->id == reply->trans_id)
+ break;
+ }
+ mutex_unlock(&instance->lock);
+
+ if (&cmd->head == &instance->rsp_list) {
+ //pr_debug("%s: received response %u, throw away...",
+ pr_err("%s: received response %u, throw away...",
+ __func__,
+ reply->trans_id);
+ } else if (reply_len > sizeof(cmd->msg)) {
+ pr_err("%s: reply too big (%u) %u, throw away...",
+ __func__, reply_len,
+ reply->trans_id);
+ } else {
+ memcpy(cmd->msg, reply,
+ reply_len);
+ complete(&cmd->cmplt);
+ }
+}
+
+static int vc_sm_cma_vchi_videocore_io(void *arg)
+{
+ struct sm_instance *instance = arg;
+ struct sm_cmd_rsp_blk *cmd = NULL, *cmd_tmp;
+ struct vc_sm_result_t *reply;
+ u32 reply_len;
+ s32 status;
+ int svc_use = 1;
+
+ while (1) {
+ if (svc_use)
+ vchi_service_release(instance->vchi_handle[0]);
+ svc_use = 0;
+
+ if (wait_for_completion_interruptible(&instance->io_cmplt))
+ continue;
+
+ vchi_service_use(instance->vchi_handle[0]);
+ svc_use = 1;
+
+ do {
+ /*
+ * Get new command and move it to response list
+ */
+ mutex_lock(&instance->lock);
+ if (list_empty(&instance->cmd_list)) {
+ /* no more commands to process */
+ mutex_unlock(&instance->lock);
+ break;
+ }
+ cmd = list_first_entry(&instance->cmd_list,
+ struct sm_cmd_rsp_blk, head);
+ list_move(&cmd->head, &instance->rsp_list);
+ cmd->sent = 1;
+ mutex_unlock(&instance->lock);
+
+ /* Send the command */
+ status =
+ bcm2835_vchi_msg_queue(instance->vchi_handle[0],
+ cmd->msg, cmd->length);
+ if (status) {
+ pr_err("%s: failed to queue message (%d)",
+ __func__, status);
+ }
+
+ /* If no reply is needed then we're done */
+ if (!cmd->wait) {
+ mutex_lock(&instance->lock);
+ list_del(&cmd->head);
+ mutex_unlock(&instance->lock);
+ vc_vchi_cmd_delete(instance, cmd);
+ continue;
+ }
+
+ if (status) {
+ complete(&cmd->cmplt);
+ continue;
+ }
+
+ } while (1);
+
+ while (!vchi_msg_peek(instance->vchi_handle[0], (void **)&reply,
+ &reply_len, VCHI_FLAGS_NONE)) {
+ if (reply->trans_id & 0x80000000) {
+ /* Async event or cmd from the VPU */
+ if (instance->vpu_event)
+ instance->vpu_event(instance, reply,
+ reply_len);
+ } else {
+ vc_sm_cma_vchi_rx_ack(instance, cmd, reply,
+ reply_len);
+ }
+
+ vchi_msg_remove(instance->vchi_handle[0]);
+ }
+
+ /* Go through the dead list and free them */
+ mutex_lock(&instance->lock);
+ list_for_each_entry_safe(cmd, cmd_tmp, &instance->dead_list,
+ head) {
+ list_del(&cmd->head);
+ vc_vchi_cmd_delete(instance, cmd);
+ }
+ mutex_unlock(&instance->lock);
+ }
+
+ return 0;
+}
+
+static void vc_sm_cma_vchi_callback(void *param,
+ const enum vchi_callback_reason reason,
+ void *msg_handle)
+{
+ struct sm_instance *instance = param;
+
+ (void)msg_handle;
+
+ switch (reason) {
+ case VCHI_CALLBACK_MSG_AVAILABLE:
+ complete(&instance->io_cmplt);
+ break;
+
+ case VCHI_CALLBACK_SERVICE_CLOSED:
+ pr_info("%s: service CLOSED!!", __func__);
+ default:
+ break;
+ }
+}
+
+struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle *vchi_instance,
+ unsigned int num_connections,
+ vpu_event_cb vpu_event)
+{
+ u32 i;
+ struct sm_instance *instance;
+ int status;
+
+ pr_debug("%s: start", __func__);
+
+ if (num_connections > VCHI_MAX_NUM_CONNECTIONS) {
+ pr_err("%s: unsupported number of connections %u (max=%u)",
+ __func__, num_connections, VCHI_MAX_NUM_CONNECTIONS);
+
+ goto err_null;
+ }
+ /* Allocate memory for this instance */
+ instance = kzalloc(sizeof(*instance), GFP_KERNEL);
+
+ /* Misc initialisations */
+ mutex_init(&instance->lock);
+ init_completion(&instance->io_cmplt);
+ INIT_LIST_HEAD(&instance->cmd_list);
+ INIT_LIST_HEAD(&instance->rsp_list);
+ INIT_LIST_HEAD(&instance->dead_list);
+ INIT_LIST_HEAD(&instance->free_list);
+ sema_init(&instance->free_sema, SM_MAX_NUM_CMD_RSP_BLKS);
+ mutex_init(&instance->free_lock);
+ for (i = 0; i < SM_MAX_NUM_CMD_RSP_BLKS; i++) {
+ init_completion(&instance->free_blk[i].cmplt);
+ list_add(&instance->free_blk[i].head, &instance->free_list);
+ }
+
+ /* Open the VCHI service connections */
+ instance->num_connections = num_connections;
+ for (i = 0; i < num_connections; i++) {
+ struct service_creation params = {
+ .version = VCHI_VERSION_EX(VC_SM_VER, VC_SM_MIN_VER),
+ .service_id = VC_SM_SERVER_NAME,
+ .callback = vc_sm_cma_vchi_callback,
+ .callback_param = instance,
+ };
+
+ status = vchi_service_open(vchi_instance,
+ ¶ms, &instance->vchi_handle[i]);
+ if (status) {
+ pr_err("%s: failed to open VCHI service (%d)",
+ __func__, status);
+
+ goto err_close_services;
+ }
+ }
+
+ /* Create the thread which takes care of all io to/from videoocore. */
+ instance->io_thread = kthread_create(&vc_sm_cma_vchi_videocore_io,
+ (void *)instance, "SMIO");
+ if (!instance->io_thread) {
+ pr_err("%s: failed to create SMIO thread", __func__);
+
+ goto err_close_services;
+ }
+ instance->vpu_event = vpu_event;
+ set_user_nice(instance->io_thread, -10);
+ wake_up_process(instance->io_thread);
+
+ pr_debug("%s: success - instance %p", __func__, instance);
+ return instance;
+
+err_close_services:
+ for (i = 0; i < instance->num_connections; i++) {
+ if (instance->vchi_handle[i])
+ vchi_service_close(instance->vchi_handle[i]);
+ }
+ kfree(instance);
+err_null:
+ pr_debug("%s: FAILED", __func__);
+ return NULL;
+}
+
+int vc_sm_cma_vchi_stop(struct sm_instance **handle)
+{
+ struct sm_instance *instance;
+ u32 i;
+
+ if (!handle) {
+ pr_err("%s: invalid pointer to handle %p", __func__, handle);
+ goto lock;
+ }
+
+ if (!*handle) {
+ pr_err("%s: invalid handle %p", __func__, *handle);
+ goto lock;
+ }
+
+ instance = *handle;
+
+ /* Close all VCHI service connections */
+ for (i = 0; i < instance->num_connections; i++) {
+ s32 success;
+
+ vchi_service_use(instance->vchi_handle[i]);
+
+ success = vchi_service_close(instance->vchi_handle[i]);
+ }
+
+ kfree(instance);
+
+ *handle = NULL;
+ return 0;
+
+lock:
+ return -EINVAL;
+}
+
+static int vc_sm_cma_vchi_send_msg(struct sm_instance *handle,
+ enum vc_sm_msg_type msg_id, void *msg,
+ u32 msg_size, void *result, u32 result_size,
+ u32 *cur_trans_id, u8 wait_reply)
+{
+ int status = 0;
+ struct sm_instance *instance = handle;
+ struct sm_cmd_rsp_blk *cmd_blk;
+
+ if (!handle) {
+ pr_err("%s: invalid handle", __func__);
+ return -EINVAL;
+ }
+ if (!msg) {
+ pr_err("%s: invalid msg pointer", __func__);
+ return -EINVAL;
+ }
+
+ cmd_blk =
+ vc_vchi_cmd_create(instance, msg_id, msg, msg_size, wait_reply);
+ if (!cmd_blk) {
+ pr_err("[%s]: failed to allocate global tracking resource",
+ __func__);
+ return -ENOMEM;
+ }
+
+ if (cur_trans_id)
+ *cur_trans_id = cmd_blk->id;
+
+ mutex_lock(&instance->lock);
+ list_add_tail(&cmd_blk->head, &instance->cmd_list);
+ mutex_unlock(&instance->lock);
+ complete(&instance->io_cmplt);
+
+ if (!wait_reply)
+ /* We're done */
+ return 0;
+
+ /* Wait for the response */
+ if (wait_for_completion_interruptible(&cmd_blk->cmplt)) {
+ mutex_lock(&instance->lock);
+ if (!cmd_blk->sent) {
+ list_del(&cmd_blk->head);
+ mutex_unlock(&instance->lock);
+ vc_vchi_cmd_delete(instance, cmd_blk);
+ return -ENXIO;
+ }
+
+ list_move(&cmd_blk->head, &instance->dead_list);
+ mutex_unlock(&instance->lock);
+ complete(&instance->io_cmplt);
+ return -EINTR; /* We're done */
+ }
+
+ if (result && result_size) {
+ memcpy(result, cmd_blk->msg, result_size);
+ } else {
+ struct vc_sm_result_t *res =
+ (struct vc_sm_result_t *)cmd_blk->msg;
+ status = (res->success == 0) ? 0 : -ENXIO;
+ }
+
+ mutex_lock(&instance->lock);
+ list_del(&cmd_blk->head);
+ mutex_unlock(&instance->lock);
+ vc_vchi_cmd_delete(instance, cmd_blk);
+ return status;
+}
+
+int vc_sm_cma_vchi_free(struct sm_instance *handle, struct vc_sm_free_t *msg,
+ u32 *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_FREE,
+ msg, sizeof(*msg), 0, 0, cur_trans_id, 0);
+}
+
+int vc_sm_cma_vchi_import(struct sm_instance *handle, struct vc_sm_import *msg,
+ struct vc_sm_import_result *result, u32 *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_IMPORT,
+ msg, sizeof(*msg), result, sizeof(*result),
+ cur_trans_id, 1);
+}
+
+int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
+ struct vc_sm_version *msg,
+ struct vc_sm_result_t *result,
+ u32 *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_CLIENT_VERSION,
+ //msg, sizeof(*msg), result, sizeof(*result),
+ //cur_trans_id, 1);
+ msg, sizeof(*msg), NULL, 0,
+ cur_trans_id, 0);
+}
+
+int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
+ struct vc_sm_vc_mem_request_result *msg,
+ uint32_t *cur_trans_id)
+{
+ return vc_sm_cma_vchi_send_msg(handle,
+ VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
+ msg, sizeof(*msg), 0, 0, cur_trans_id,
+ 0);
+}
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
new file mode 100644
index 000000000000..e8db34bd1e91
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
+ *
+ * Based on vmcs_sm driver from Broadcom Corporation.
+ *
+ */
+
+#ifndef __VC_SM_CMA_VCHI_H__INCLUDED__
+#define __VC_SM_CMA_VCHI_H__INCLUDED__
+
+#include "interface/vchi/vchi.h"
+
+#include "vc_sm_defs.h"
+
+/*
+ * Forward declare.
+ */
+struct sm_instance;
+
+typedef void (*vpu_event_cb)(struct sm_instance *instance,
+ struct vc_sm_result_t *reply, int reply_len);
+
+/*
+ * Initialize the shared memory service, opens up vchi connection to talk to it.
+ */
+struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle *vchi_instance,
+ unsigned int num_connections,
+ vpu_event_cb vpu_event);
+
+/*
+ * Terminates the shared memory service.
+ */
+int vc_sm_cma_vchi_stop(struct sm_instance **handle);
+
+/*
+ * Ask the shared memory service to free up some memory that was previously
+ * allocated by the vc_sm_cma_vchi_alloc function call.
+ */
+int vc_sm_cma_vchi_free(struct sm_instance *handle, struct vc_sm_free_t *msg,
+ u32 *cur_trans_id);
+
+/*
+ * Import a contiguous block of memory and wrap it in a GPU MEM_HANDLE_T.
+ */
+int vc_sm_cma_vchi_import(struct sm_instance *handle, struct vc_sm_import *msg,
+ struct vc_sm_import_result *result,
+ u32 *cur_trans_id);
+
+int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
+ struct vc_sm_version *msg,
+ struct vc_sm_result_t *result,
+ u32 *cur_trans_id);
+
+int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
+ struct vc_sm_vc_mem_request_result *msg,
+ uint32_t *cur_trans_id);
+
+#endif /* __VC_SM_CMA_VCHI_H__INCLUDED__ */
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
new file mode 100644
index 000000000000..8a0d1f6dbfe8
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
@@ -0,0 +1,300 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ *
+ * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom Corporation.
+ * All IPC messages are copied across to this file, even if the vc-sm-cma
+ * driver is not currently using them.
+ *
+ ****************************************************************************
+ */
+
+#ifndef __VC_SM_DEFS_H__INCLUDED__
+#define __VC_SM_DEFS_H__INCLUDED__
+
+/* FourCC code used for VCHI connection */
+#define VC_SM_SERVER_NAME MAKE_FOURCC("SMEM")
+
+/* Maximum message length */
+#define VC_SM_MAX_MSG_LEN (sizeof(union vc_sm_msg_union_t) + \
+ sizeof(struct vc_sm_msg_hdr_t))
+#define VC_SM_MAX_RSP_LEN (sizeof(union vc_sm_msg_union_t))
+
+/* Resource name maximum size */
+#define VC_SM_RESOURCE_NAME 32
+
+/*
+ * Version to be reported to the VPU
+ * VPU assumes 0 (aka 1) which does not require the released callback, nor
+ * expect the client to handle VC_MEM_REQUESTS.
+ * Version 2 requires the released callback, and must support VC_MEM_REQUESTS.
+ */
+#define VC_SM_PROTOCOL_VERSION 2
+
+enum vc_sm_msg_type {
+ /* Message types supported for HOST->VC direction */
+
+ /* Allocate shared memory block */
+ VC_SM_MSG_TYPE_ALLOC,
+ /* Lock allocated shared memory block */
+ VC_SM_MSG_TYPE_LOCK,
+ /* Unlock allocated shared memory block */
+ VC_SM_MSG_TYPE_UNLOCK,
+ /* Unlock allocated shared memory block, do not answer command */
+ VC_SM_MSG_TYPE_UNLOCK_NOANS,
+ /* Free shared memory block */
+ VC_SM_MSG_TYPE_FREE,
+ /* Resize a shared memory block */
+ VC_SM_MSG_TYPE_RESIZE,
+ /* Walk the allocated shared memory block(s) */
+ VC_SM_MSG_TYPE_WALK_ALLOC,
+
+ /* A previously applied action will need to be reverted */
+ VC_SM_MSG_TYPE_ACTION_CLEAN,
+
+ /*
+ * Import a physical address and wrap into a MEM_HANDLE_T.
+ * Release with VC_SM_MSG_TYPE_FREE.
+ */
+ VC_SM_MSG_TYPE_IMPORT,
+ /*
+ *Tells VC the protocol version supported by this client.
+ * 2 supports the async/cmd messages from the VPU for final release
+ * of memory, and for VC allocations.
+ */
+ VC_SM_MSG_TYPE_CLIENT_VERSION,
+ /* Response to VC request for memory */
+ VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
+
+ /*
+ * Asynchronous/cmd messages supported for VC->HOST direction.
+ * Signalled by setting the top bit in vc_sm_result_t trans_id.
+ */
+
+ /*
+ * VC has finished with an imported memory allocation.
+ * Release any Linux reference counts on the underlying block.
+ */
+ VC_SM_MSG_TYPE_RELEASED,
+ /* VC request for memory */
+ VC_SM_MSG_TYPE_VC_MEM_REQUEST,
+
+ VC_SM_MSG_TYPE_MAX
+};
+
+/* Type of memory to be allocated */
+enum vc_sm_alloc_type_t {
+ VC_SM_ALLOC_CACHED,
+ VC_SM_ALLOC_NON_CACHED,
+};
+
+/* Message header for all messages in HOST->VC direction */
+struct vc_sm_msg_hdr_t {
+ u32 type;
+ u32 trans_id;
+ u8 body[0];
+
+};
+
+/* Request to allocate memory (HOST->VC) */
+struct vc_sm_alloc_t {
+ /* type of memory to allocate */
+ enum vc_sm_alloc_type_t type;
+ /* byte amount of data to allocate per unit */
+ u32 base_unit;
+ /* number of unit to allocate */
+ u32 num_unit;
+ /* alignment to be applied on allocation */
+ u32 alignment;
+ /* identity of who allocated this block */
+ u32 allocator;
+ /* resource name (for easier tracking on vc side) */
+ char name[VC_SM_RESOURCE_NAME];
+
+};
+
+/* Result of a requested memory allocation (VC->HOST) */
+struct vc_sm_alloc_result_t {
+ /* Transaction identifier */
+ u32 trans_id;
+
+ /* Resource handle */
+ u32 res_handle;
+ /* Pointer to resource buffer */
+ u32 res_mem;
+ /* Resource base size (bytes) */
+ u32 res_base_size;
+ /* Resource number */
+ u32 res_num;
+
+};
+
+/* Request to free a previously allocated memory (HOST->VC) */
+struct vc_sm_free_t {
+ /* Resource handle (returned from alloc) */
+ u32 res_handle;
+ /* Resource buffer (returned from alloc) */
+ u32 res_mem;
+
+};
+
+/* Request to lock a previously allocated memory (HOST->VC) */
+struct vc_sm_lock_unlock_t {
+ /* Resource handle (returned from alloc) */
+ u32 res_handle;
+ /* Resource buffer (returned from alloc) */
+ u32 res_mem;
+
+};
+
+/* Request to resize a previously allocated memory (HOST->VC) */
+struct vc_sm_resize_t {
+ /* Resource handle (returned from alloc) */
+ u32 res_handle;
+ /* Resource buffer (returned from alloc) */
+ u32 res_mem;
+ /* Resource *new* size requested (bytes) */
+ u32 res_new_size;
+
+};
+
+/* Result of a requested memory lock (VC->HOST) */
+struct vc_sm_lock_result_t {
+ /* Transaction identifier */
+ u32 trans_id;
+
+ /* Resource handle */
+ u32 res_handle;
+ /* Pointer to resource buffer */
+ u32 res_mem;
+ /*
+ * Pointer to former resource buffer if the memory
+ * was reallocated
+ */
+ u32 res_old_mem;
+
+};
+
+/* Generic result for a request (VC->HOST) */
+struct vc_sm_result_t {
+ /* Transaction identifier */
+ u32 trans_id;
+
+ s32 success;
+
+};
+
+/* Request to revert a previously applied action (HOST->VC) */
+struct vc_sm_action_clean_t {
+ /* Action of interest */
+ enum vc_sm_msg_type res_action;
+ /* Transaction identifier for the action of interest */
+ u32 action_trans_id;
+
+};
+
+/* Request to remove all data associated with a given allocator (HOST->VC) */
+struct vc_sm_free_all_t {
+ /* Allocator identifier */
+ u32 allocator;
+};
+
+/* Request to import memory (HOST->VC) */
+struct vc_sm_import {
+ /* type of memory to allocate */
+ enum vc_sm_alloc_type_t type;
+ /* pointer to the VC (ie physical) address of the allocated memory */
+ u32 addr;
+ /* size of buffer */
+ u32 size;
+ /* opaque handle returned in RELEASED messages */
+ u32 kernel_id;
+ /* Allocator identifier */
+ u32 allocator;
+ /* resource name (for easier tracking on vc side) */
+ char name[VC_SM_RESOURCE_NAME];
+};
+
+/* Result of a requested memory import (VC->HOST) */
+struct vc_sm_import_result {
+ /* Transaction identifier */
+ u32 trans_id;
+
+ /* Resource handle */
+ u32 res_handle;
+};
+
+/* Notification that VC has finished with an allocation (VC->HOST) */
+struct vc_sm_released {
+ /* cmd type / trans_id */
+ u32 cmd;
+
+ /* pointer to the VC (ie physical) address of the allocated memory */
+ u32 addr;
+ /* size of buffer */
+ u32 size;
+ /* opaque handle returned in RELEASED messages */
+ u32 kernel_id;
+ u32 vc_handle;
+};
+
+/*
+ * Client informing VC as to the protocol version it supports.
+ * >=2 requires the released callback, and supports VC asking for memory.
+ * Failure means that the firmware doesn't support this call, and therefore the
+ * client should either fail, or NOT rely on getting the released callback.
+ */
+struct vc_sm_version {
+ u32 version;
+};
+
+/* Request FROM VideoCore for some memory */
+struct vc_sm_vc_mem_request {
+ /* cmd type */
+ u32 cmd;
+
+ /* trans_id (from VPU) */
+ u32 trans_id;
+ /* size of buffer */
+ u32 size;
+ /* alignment of buffer */
+ u32 align;
+ /* resource name (for easier tracking) */
+ char name[VC_SM_RESOURCE_NAME];
+ /* VPU handle for the resource */
+ u32 vc_handle;
+};
+
+/* Response from the kernel to provide the VPU with some memory */
+struct vc_sm_vc_mem_request_result {
+ /* Transaction identifier for the VPU */
+ u32 trans_id;
+ /* pointer to the physical address of the allocated memory */
+ u32 addr;
+ /* opaque handle returned in RELEASED messages */
+ u32 kernel_id;
+};
+
+/* Union of ALL messages */
+union vc_sm_msg_union_t {
+ struct vc_sm_alloc_t alloc;
+ struct vc_sm_alloc_result_t alloc_result;
+ struct vc_sm_free_t free;
+ struct vc_sm_lock_unlock_t lock_unlock;
+ struct vc_sm_action_clean_t action_clean;
+ struct vc_sm_resize_t resize;
+ struct vc_sm_lock_result_t lock_result;
+ struct vc_sm_result_t result;
+ struct vc_sm_free_all_t free_all;
+ struct vc_sm_import import;
+ struct vc_sm_import_result import_result;
+ struct vc_sm_version version;
+ struct vc_sm_released released;
+ struct vc_sm_vc_mem_request vc_request;
+ struct vc_sm_vc_mem_request_result vc_request_result;
+};
+
+#endif /* __VC_SM_DEFS_H__INCLUDED__ */
diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
new file mode 100644
index 000000000000..988fdd967922
--- /dev/null
+++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
@@ -0,0 +1,28 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+/*
+ * VideoCore Shared Memory CMA allocator
+ *
+ * Copyright: 2018, Raspberry Pi (Trading) Ltd
+ *
+ * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom Corporation.
+ *
+ */
+
+#ifndef __VC_SM_KNL_H__INCLUDED__
+#define __VC_SM_KNL_H__INCLUDED__
+
+#if !defined(__KERNEL__)
+#error "This interface is for kernel use only..."
+#endif
+
+/* Free a previously allocated or imported shared memory handle and block. */
+int vc_sm_cma_free(void *handle);
+
+/* Get an internal resource handle mapped from the external one. */
+int vc_sm_cma_int_handle(void *handle);
+
+/* Import a block of memory into the GPU space. */
+int vc_sm_cma_import_dmabuf(struct dma_buf *dmabuf, void **handle);
+
+#endif /* __VC_SM_KNL_H__INCLUDED__ */
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-04 9:25 ` [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver Laurent Pinchart
@ 2020-05-05 2:38 ` kbuild test robot
2020-05-05 12:17 ` kbuild test robot
` (3 subsequent siblings)
4 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 2:38 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: kbuild-all, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 2428 bytes --]
Hi Laurent,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200504]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
config: c6x-allyesconfig (attached as .config)
compiler: c6x-elf-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=c6x
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c: In function 'cache_op_to_func':
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:10: error: 'dmac_flush_range' undeclared (first use in this function)
1265 | return dmac_flush_range;
| ^~~~~~~~~~~~~~~~
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:10: note: each undeclared identifier is reported only once for each function it appears in
vim +/dmac_flush_range +1265 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
1252
1253 #ifndef CONFIG_ARM64
1254 /* Converts VCSM_CACHE_OP_* to an operating function. */
1255 static void (*cache_op_to_func(const unsigned int cache_op))
1256 (const void*, const void*)
1257 {
1258 switch (cache_op) {
1259 case VC_SM_CACHE_OP_NOP:
1260 return NULL;
1261
1262 case VC_SM_CACHE_OP_INV:
1263 case VC_SM_CACHE_OP_CLEAN:
1264 case VC_SM_CACHE_OP_FLUSH:
> 1265 return dmac_flush_range;
1266
1267 default:
1268 pr_err("[%s]: Invalid cache_op: 0x%08x\n", __func__, cache_op);
1269 return NULL;
1270 }
1271 }
1272
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 52571 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
@ 2020-05-05 2:38 ` kbuild test robot
0 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 2:38 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2489 bytes --]
Hi Laurent,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200504]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
config: c6x-allyesconfig (attached as .config)
compiler: c6x-elf-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=c6x
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c: In function 'cache_op_to_func':
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:10: error: 'dmac_flush_range' undeclared (first use in this function)
1265 | return dmac_flush_range;
| ^~~~~~~~~~~~~~~~
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:10: note: each undeclared identifier is reported only once for each function it appears in
vim +/dmac_flush_range +1265 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
1252
1253 #ifndef CONFIG_ARM64
1254 /* Converts VCSM_CACHE_OP_* to an operating function. */
1255 static void (*cache_op_to_func(const unsigned int cache_op))
1256 (const void*, const void*)
1257 {
1258 switch (cache_op) {
1259 case VC_SM_CACHE_OP_NOP:
1260 return NULL;
1261
1262 case VC_SM_CACHE_OP_INV:
1263 case VC_SM_CACHE_OP_CLEAN:
1264 case VC_SM_CACHE_OP_FLUSH:
> 1265 return dmac_flush_range;
1266
1267 default:
1268 pr_err("[%s]: Invalid cache_op: 0x%08x\n", __func__, cache_op);
1269 return NULL;
1270 }
1271 }
1272
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 52571 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-04 9:25 ` [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver Laurent Pinchart
@ 2020-05-05 12:17 ` kbuild test robot
2020-05-05 12:17 ` kbuild test robot
` (3 subsequent siblings)
4 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 12:17 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: kbuild-all, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 2428 bytes --]
Hi Laurent,
I love your patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
config: sparc-allyesconfig (attached as .config)
compiler: sparc64-linux-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=sparc
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c: In function 'cache_op_to_func':
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:10: error: 'dmac_flush_range' undeclared (first use in this function)
1265 | return dmac_flush_range;
| ^~~~~~~~~~~~~~~~
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:10: note: each undeclared identifier is reported only once for each function it appears in
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c: At top level:
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1468:3: error: unknown type name 'compat_uptr_t'
1468 | compat_uptr_t start_address;
| ^~~~~~~~~~~~~
vim +/compat_uptr_t +1468 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
1461
1462 #ifdef CONFIG_COMPAT
1463 struct vc_sm_cma_ioctl_clean_invalid2_32 {
1464 u32 op_count;
1465 struct vc_sm_cma_ioctl_clean_invalid_block_32 {
1466 u16 invalidate_mode;
1467 u16 block_count;
> 1468 compat_uptr_t start_address;
1469 u32 block_size;
1470 u32 inter_block_stride;
1471 } s[0];
1472 };
1473
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 62604 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-04 9:25 ` [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver Laurent Pinchart
@ 2020-05-06 3:05 ` kbuild test robot
2020-05-05 12:17 ` kbuild test robot
` (3 subsequent siblings)
4 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-06 3:05 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: kbuild-all, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Laurent,
I love your patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
reproduce:
# apt-get install sparse
# sparse version: v0.6.1-191-gc51a0382-dirty
make ARCH=x86_64 allmodconfig
make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'
:::::: branch date: 17 hours ago
:::::: commit date: 17 hours ago
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
sparse warnings: (new ones prefixed by >>)
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm.h:60:21: sparse: sparse: dubious one-bit signed bitfield
drivers/staging/vc04_services/vc-sm-cma/vc_sm.h:61:23: sparse: sparse: dubious one-bit signed bitfield
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:24: sparse: sparse: undefined identifier 'dmac_flush_range'
--
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c:474:55: sparse: sparse: Using plain integer as NULL pointer
drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c:503:59: sparse: sparse: Using plain integer as NULL pointer
# https://github.com/0day-ci/linux/commit/5f041a24bca6d9babfbb0e35b8e2f7cea4cf933b
git remote add linux-review https://github.com/0day-ci/linux
git remote update linux-review
git checkout 5f041a24bca6d9babfbb0e35b8e2f7cea4cf933b
vim +60 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
5f041a24bca6d9 Dave Stevenson 2020-05-04 43
5f041a24bca6d9 Dave Stevenson 2020-05-04 44 struct vc_sm_buffer {
5f041a24bca6d9 Dave Stevenson 2020-05-04 45 struct list_head global_buffer_list; /* Global list of buffers. */
5f041a24bca6d9 Dave Stevenson 2020-05-04 46
5f041a24bca6d9 Dave Stevenson 2020-05-04 47 /* Index in the kernel_id idr so that we can find the
5f041a24bca6d9 Dave Stevenson 2020-05-04 48 * mmal_msg_context again when servicing the VCHI reply.
5f041a24bca6d9 Dave Stevenson 2020-05-04 49 */
5f041a24bca6d9 Dave Stevenson 2020-05-04 50 int kernel_id;
5f041a24bca6d9 Dave Stevenson 2020-05-04 51
5f041a24bca6d9 Dave Stevenson 2020-05-04 52 size_t size;
5f041a24bca6d9 Dave Stevenson 2020-05-04 53
5f041a24bca6d9 Dave Stevenson 2020-05-04 54 /* Lock over all the following state for this buffer */
5f041a24bca6d9 Dave Stevenson 2020-05-04 55 struct mutex lock;
5f041a24bca6d9 Dave Stevenson 2020-05-04 56 struct list_head attachments;
5f041a24bca6d9 Dave Stevenson 2020-05-04 57
5f041a24bca6d9 Dave Stevenson 2020-05-04 58 char name[VC_SM_MAX_NAME_LEN];
5f041a24bca6d9 Dave Stevenson 2020-05-04 59
5f041a24bca6d9 Dave Stevenson 2020-05-04 @60 int in_use:1; /* Kernel is still using this resource */
5f041a24bca6d9 Dave Stevenson 2020-05-04 61 int imported:1; /* Imported dmabuf */
5f041a24bca6d9 Dave Stevenson 2020-05-04 62
5f041a24bca6d9 Dave Stevenson 2020-05-04 63 enum vc_sm_vpu_mapping_state vpu_state;
5f041a24bca6d9 Dave Stevenson 2020-05-04 64 u32 vc_handle; /* VideoCore handle for this buffer */
5f041a24bca6d9 Dave Stevenson 2020-05-04 65 int vpu_allocated; /*
5f041a24bca6d9 Dave Stevenson 2020-05-04 66 * The VPU made this allocation. Release the
5f041a24bca6d9 Dave Stevenson 2020-05-04 67 * local dma_buf when the VPU releases the
5f041a24bca6d9 Dave Stevenson 2020-05-04 68 * resource.
5f041a24bca6d9 Dave Stevenson 2020-05-04 69 */
5f041a24bca6d9 Dave Stevenson 2020-05-04 70
5f041a24bca6d9 Dave Stevenson 2020-05-04 71 /* DMABUF related fields */
5f041a24bca6d9 Dave Stevenson 2020-05-04 72 struct dma_buf *dma_buf;
5f041a24bca6d9 Dave Stevenson 2020-05-04 73 dma_addr_t dma_addr;
5f041a24bca6d9 Dave Stevenson 2020-05-04 74 void *cookie;
5f041a24bca6d9 Dave Stevenson 2020-05-04 75
5f041a24bca6d9 Dave Stevenson 2020-05-04 76 struct vc_sm_privdata_t *private;
5f041a24bca6d9 Dave Stevenson 2020-05-04 77
5f041a24bca6d9 Dave Stevenson 2020-05-04 78 union {
5f041a24bca6d9 Dave Stevenson 2020-05-04 79 struct vc_sm_alloc_data alloc;
5f041a24bca6d9 Dave Stevenson 2020-05-04 80 struct vc_sm_imported import;
5f041a24bca6d9 Dave Stevenson 2020-05-04 81 };
5f041a24bca6d9 Dave Stevenson 2020-05-04 82 };
5f041a24bca6d9 Dave Stevenson 2020-05-04 83
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
@ 2020-05-06 3:05 ` kbuild test robot
0 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-06 3:05 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4997 bytes --]
Hi Laurent,
I love your patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
reproduce:
# apt-get install sparse
# sparse version: v0.6.1-191-gc51a0382-dirty
make ARCH=x86_64 allmodconfig
make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'
:::::: branch date: 17 hours ago
:::::: commit date: 17 hours ago
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
sparse warnings: (new ones prefixed by >>)
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm.h:60:21: sparse: sparse: dubious one-bit signed bitfield
drivers/staging/vc04_services/vc-sm-cma/vc_sm.h:61:23: sparse: sparse: dubious one-bit signed bitfield
drivers/staging/vc04_services/vc-sm-cma/vc_sm.c:1265:24: sparse: sparse: undefined identifier 'dmac_flush_range'
--
>> drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c:474:55: sparse: sparse: Using plain integer as NULL pointer
drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c:503:59: sparse: sparse: Using plain integer as NULL pointer
# https://github.com/0day-ci/linux/commit/5f041a24bca6d9babfbb0e35b8e2f7cea4cf933b
git remote add linux-review https://github.com/0day-ci/linux
git remote update linux-review
git checkout 5f041a24bca6d9babfbb0e35b8e2f7cea4cf933b
vim +60 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
5f041a24bca6d9 Dave Stevenson 2020-05-04 43
5f041a24bca6d9 Dave Stevenson 2020-05-04 44 struct vc_sm_buffer {
5f041a24bca6d9 Dave Stevenson 2020-05-04 45 struct list_head global_buffer_list; /* Global list of buffers. */
5f041a24bca6d9 Dave Stevenson 2020-05-04 46
5f041a24bca6d9 Dave Stevenson 2020-05-04 47 /* Index in the kernel_id idr so that we can find the
5f041a24bca6d9 Dave Stevenson 2020-05-04 48 * mmal_msg_context again when servicing the VCHI reply.
5f041a24bca6d9 Dave Stevenson 2020-05-04 49 */
5f041a24bca6d9 Dave Stevenson 2020-05-04 50 int kernel_id;
5f041a24bca6d9 Dave Stevenson 2020-05-04 51
5f041a24bca6d9 Dave Stevenson 2020-05-04 52 size_t size;
5f041a24bca6d9 Dave Stevenson 2020-05-04 53
5f041a24bca6d9 Dave Stevenson 2020-05-04 54 /* Lock over all the following state for this buffer */
5f041a24bca6d9 Dave Stevenson 2020-05-04 55 struct mutex lock;
5f041a24bca6d9 Dave Stevenson 2020-05-04 56 struct list_head attachments;
5f041a24bca6d9 Dave Stevenson 2020-05-04 57
5f041a24bca6d9 Dave Stevenson 2020-05-04 58 char name[VC_SM_MAX_NAME_LEN];
5f041a24bca6d9 Dave Stevenson 2020-05-04 59
5f041a24bca6d9 Dave Stevenson 2020-05-04 @60 int in_use:1; /* Kernel is still using this resource */
5f041a24bca6d9 Dave Stevenson 2020-05-04 61 int imported:1; /* Imported dmabuf */
5f041a24bca6d9 Dave Stevenson 2020-05-04 62
5f041a24bca6d9 Dave Stevenson 2020-05-04 63 enum vc_sm_vpu_mapping_state vpu_state;
5f041a24bca6d9 Dave Stevenson 2020-05-04 64 u32 vc_handle; /* VideoCore handle for this buffer */
5f041a24bca6d9 Dave Stevenson 2020-05-04 65 int vpu_allocated; /*
5f041a24bca6d9 Dave Stevenson 2020-05-04 66 * The VPU made this allocation. Release the
5f041a24bca6d9 Dave Stevenson 2020-05-04 67 * local dma_buf when the VPU releases the
5f041a24bca6d9 Dave Stevenson 2020-05-04 68 * resource.
5f041a24bca6d9 Dave Stevenson 2020-05-04 69 */
5f041a24bca6d9 Dave Stevenson 2020-05-04 70
5f041a24bca6d9 Dave Stevenson 2020-05-04 71 /* DMABUF related fields */
5f041a24bca6d9 Dave Stevenson 2020-05-04 72 struct dma_buf *dma_buf;
5f041a24bca6d9 Dave Stevenson 2020-05-04 73 dma_addr_t dma_addr;
5f041a24bca6d9 Dave Stevenson 2020-05-04 74 void *cookie;
5f041a24bca6d9 Dave Stevenson 2020-05-04 75
5f041a24bca6d9 Dave Stevenson 2020-05-04 76 struct vc_sm_privdata_t *private;
5f041a24bca6d9 Dave Stevenson 2020-05-04 77
5f041a24bca6d9 Dave Stevenson 2020-05-04 78 union {
5f041a24bca6d9 Dave Stevenson 2020-05-04 79 struct vc_sm_alloc_data alloc;
5f041a24bca6d9 Dave Stevenson 2020-05-04 80 struct vc_sm_imported import;
5f041a24bca6d9 Dave Stevenson 2020-05-04 81 };
5f041a24bca6d9 Dave Stevenson 2020-05-04 82 };
5f041a24bca6d9 Dave Stevenson 2020-05-04 83
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-04 9:25 ` [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver Laurent Pinchart
` (2 preceding siblings ...)
2020-05-06 3:05 ` kbuild test robot
@ 2020-05-06 18:04 ` Nicolas Saenz Julienne
2020-05-06 19:24 ` Dave Stevenson
2020-05-11 18:42 ` Nicolas Saenz Julienne
4 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-06 18:04 UTC (permalink / raw)
To: Laurent Pinchart, linux-media, dave.stevenson
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
Hi Laurent, Dave,
On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> Add Broadcom VideoCore Shared Memory support.
>
> This new driver allows contiguous memory blocks to be imported
> into the VideoCore VPU memory map, and manages the lifetime of
> those objects, only releasing the source dmabuf once the VPU has
> confirmed it has finished with it.
>
I'm still digesting all this, but a question came up, who is using the
ioctls?
Regards,
Nicolas
> Driver upported from the RaspberryPi BSP at revision:
> 890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when
> counting transactions")
> forward ported to recent mainline kernel version.
>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
> drivers/staging/vc04_services/Kconfig | 2 +
> drivers/staging/vc04_services/Makefile | 1 +
> .../include/linux/broadcom/vc_sm_cma_ioctl.h | 114 ++
> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> .../staging/vc04_services/vc-sm-cma/Makefile | 13 +
> drivers/staging/vc04_services/vc-sm-cma/TODO | 1 +
> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 1732
> +++++++++++++++++
> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 84 +
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 505 +++++
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 300 +++
> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> 12 files changed, 2853 insertions(+)
> create mode 100644
> drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.
> h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/TODO
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-
> cma/vc_sm_cma_vchi.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-
> cma/vc_sm_cma_vchi.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-
> cma/vc_sm_defs.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-
> cma/vc_sm_knl.h
>
> diff --git a/drivers/staging/vc04_services/Kconfig
> b/drivers/staging/vc04_services/Kconfig
> index 6baf9dd57f1f..b9f1f019ebd8 100644
> --- a/drivers/staging/vc04_services/Kconfig
> +++ b/drivers/staging/vc04_services/Kconfig
> @@ -23,5 +23,7 @@ source "drivers/staging/vc04_services/bcm2835-
> audio/Kconfig"
>
> source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
>
> +source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
> +
> endif
>
> diff --git a/drivers/staging/vc04_services/Makefile
> b/drivers/staging/vc04_services/Makefile
> index 54d9e2f31916..6e1abf494c1a 100644
> --- a/drivers/staging/vc04_services/Makefile
> +++ b/drivers/staging/vc04_services/Makefile
> @@ -12,6 +12,7 @@ vchiq-objs := \
>
> obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
> obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
> +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
>
> ccflags-y += -D__VCCOREVER__=0x04000000
>
> diff --git
> a/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> l.h
> b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> l.h
> new file mode 100644
> index 000000000000..107460ad1be3
> --- /dev/null
> +++
> b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> l.h
> @@ -0,0 +1,114 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * Copyright 2019 Raspberry Pi (Trading) Ltd. All rights reserved.
> + *
> + * Based on vmcs_sm_ioctl.h Copyright Broadcom Corporation.
> + */
> +
> +#ifndef __VC_SM_CMA_IOCTL_H
> +#define __VC_SM_CMA_IOCTL_H
> +
> +/* ---- Include Files ----------------------------------------------
> ------ */
> +
> +#if defined(__KERNEL__)
> +#include <linux/types.h> /* Needed for standard types */
> +#else
> +#include <stdint.h>
> +#endif
> +
> +#include <linux/ioctl.h>
> +
> +/* ---- Constants and Types ----------------------------------------
> ------ */
> +
> +#define VC_SM_CMA_RESOURCE_NAME 32
> +#define VC_SM_CMA_RESOURCE_NAME_DEFAULT "sm-host-resource"
> +
> +/* Type define used to create unique IOCTL number */
> +#define VC_SM_CMA_MAGIC_TYPE 'J'
> +
> +/* IOCTL commands on /dev/vc-sm-cma */
> +enum vc_sm_cma_cmd_e {
> + VC_SM_CMA_CMD_ALLOC = 0x5A, /* Start at 0x5A arbitrarily
> */
> +
> + VC_SM_CMA_CMD_IMPORT_DMABUF,
> +
> + VC_SM_CMA_CMD_CLEAN_INVALID2,
> +
> + VC_SM_CMA_CMD_LAST /* Do not delete */
> +};
> +
> +/* Cache type supported, conveniently matches the user space
> definition in
> + * user-vcsm.h.
> + */
> +enum vc_sm_cma_cache_e {
> + VC_SM_CMA_CACHE_NONE,
> + VC_SM_CMA_CACHE_HOST,
> + VC_SM_CMA_CACHE_VC,
> + VC_SM_CMA_CACHE_BOTH,
> +};
> +
> +/* IOCTL Data structures */
> +struct vc_sm_cma_ioctl_alloc {
> + /* user -> kernel */
> + __u32 size;
> + __u32 num;
> + __u32 cached; /* enum vc_sm_cma_cache_e */
> + __u32 pad;
> + __u8 name[VC_SM_CMA_RESOURCE_NAME];
> +
> + /* kernel -> user */
> + __s32 handle;
> + __u32 vc_handle;
> + __u64 dma_addr;
> +};
> +
> +struct vc_sm_cma_ioctl_import_dmabuf {
> + /* user -> kernel */
> + __s32 dmabuf_fd;
> + __u32 cached; /* enum vc_sm_cma_cache_e */
> + __u8 name[VC_SM_CMA_RESOURCE_NAME];
> +
> + /* kernel -> user */
> + __s32 handle;
> + __u32 vc_handle;
> + __u32 size;
> + __u32 pad;
> + __u64 dma_addr;
> +};
> +
> +/*
> + * Cache functions to be set to struct
> vc_sm_cma_ioctl_clean_invalid2
> + * invalidate_mode.
> + */
> +#define VC_SM_CACHE_OP_NOP 0x00
> +#define VC_SM_CACHE_OP_INV 0x01
> +#define VC_SM_CACHE_OP_CLEAN 0x02
> +#define VC_SM_CACHE_OP_FLUSH 0x03
> +
> +struct vc_sm_cma_ioctl_clean_invalid2 {
> + __u32 op_count;
> + __u32 pad;
> + struct vc_sm_cma_ioctl_clean_invalid_block {
> + __u32 invalidate_mode;
> + __u32 block_count;
> + void * __user start_address;
> + __u32 block_size;
> + __u32 inter_block_stride;
> + } s[0];
> +};
> +
> +/* IOCTL numbers */
> +#define VC_SM_CMA_IOCTL_MEM_ALLOC\
> + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_ALLOC,\
> + struct vc_sm_cma_ioctl_alloc)
> +
> +#define VC_SM_CMA_IOCTL_MEM_IMPORT_DMABUF\
> + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_IMPORT_DMABUF,\
> + struct vc_sm_cma_ioctl_import_dmabuf)
> +
> +#define VC_SM_CMA_IOCTL_MEM_CLEAN_INVALID2\
> + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
> + struct vc_sm_cma_ioctl_clean_invalid2)
> +
> +#endif /* __VC_SM_CMA_IOCTL_H */
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> new file mode 100644
> index 000000000000..5ac115da6b49
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> @@ -0,0 +1,10 @@
> +config BCM_VC_SM_CMA
> + bool "VideoCore Shared Memory (CMA) driver"
> + select BCM2835_VCHIQ
> + select RBTREE
> + select DMA_SHARED_BUFFER
> + help
> + Say Y here to enable the shared memory interface that
> + supports sharing dmabufs with VideoCore.
> + This operates over the VCHIQ interface to a service
> + running on VideoCore.
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/Makefile
> b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> new file mode 100644
> index 000000000000..77d173694fbf
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> @@ -0,0 +1,13 @@
> +ccflags-y += \
> + -I$(srctree)/$(src)/../ \
> + -I$(srctree)/$(src)/../interface/vchi \
> + -I$(srctree)/$(src)/../interface/vchiq_arm\
> + -I$(srctree)/$(src)/../include
> +
> +ccflags-y += \
> + -D__VCCOREVER__=0
> +
> +vc-sm-cma-$(CONFIG_BCM_VC_SM_CMA) := \
> + vc_sm.o vc_sm_cma_vchi.o
> +
> +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma.o
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/TODO
> b/drivers/staging/vc04_services/vc-sm-cma/TODO
> new file mode 100644
> index 000000000000..ac9b5f8a7389
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/TODO
> @@ -0,0 +1 @@
> +No currently outstanding tasks except some clean-up.
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> new file mode 100644
> index 000000000000..cd5fb561debb
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> @@ -0,0 +1,1732 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * VideoCore Shared Memory driver using CMA.
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + * Dave Stevenson <dave.stevenson@raspberrypi.org>
> + *
> + * Based on vmcs_sm driver from Broadcom Corporation for some API,
> + * and taking some code for buffer allocation and dmabuf handling
> from
> + * videobuf2.
> + *
> + *
> + * This driver has 3 main uses:
> + * 1) Allocating buffers for the kernel or userspace that can be
> shared with the
> + * VPU.
> + * 2) Importing dmabufs from elsewhere for sharing with the VPU.
> + * 3) Allocating buffers for use by the VPU.
> + *
> + * In the first and second cases the native handle is a dmabuf.
> Releasing the
> + * resource inherently comes from releasing the dmabuf, and this
> will trigger
> + * unmapping on the VPU. The underlying allocation and our buffer
> structure are
> + * retained until the VPU has confirmed that it has finished with
> it.
> + *
> + * For the VPU allocations the VPU is responsible for triggering the
> release,
> + * and therefore the released message decrements the dma_buf
> refcount (with the
> + * VPU mapping having already been marked as released).
> + */
> +
> +/* ---- Include Files ----------------------------------------------
> ------- */
> +#include <linux/cdev.h>
> +#include <linux/device.h>
> +#include <linux/debugfs.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/dma-buf.h>
> +#include <linux/errno.h>
> +#include <linux/fs.h>
> +#include <linux/kernel.h>
> +#include <linux/list.h>
> +#include <linux/miscdevice.h>
> +#include <linux/module.h>
> +#include <linux/mm.h>
> +#include <linux/of_device.h>
> +#include <linux/platform_device.h>
> +#include <linux/proc_fs.h>
> +#include <linux/slab.h>
> +#include <linux/seq_file.h>
> +#include <linux/syscalls.h>
> +#include <linux/types.h>
> +#include <asm/cacheflush.h>
> +
> +#include "vchiq_connected.h"
> +#include "vc_sm_cma_vchi.h"
> +
> +#include "vc_sm.h"
> +#include "vc_sm_knl.h"
> +#include <linux/broadcom/vc_sm_cma_ioctl.h>
> +
> +/* ---- Private Constants and Types --------------------------------
> ------- */
> +
> +#define DEVICE_NAME "vcsm-cma"
> +#define DEVICE_MINOR 0
> +
> +#define VC_SM_RESOURCE_NAME_DEFAULT "sm-host-resource"
> +
> +#define VC_SM_DIR_ROOT_NAME "vcsm-cma"
> +#define VC_SM_STATE "state"
> +
> +/* Private file data associated with each opened device. */
> +struct vc_sm_privdata_t {
> + pid_t pid; /* PID of creator. */
> +
> + int restart_sys; /* Tracks restart on interrupt. */
> + enum vc_sm_msg_type int_action; /* Interrupted action. */
> + u32 int_trans_id; /* Interrupted transaction. */
> +};
> +
> +typedef int (*VC_SM_SHOW) (struct seq_file *s, void *v);
> +struct sm_pde_t {
> + VC_SM_SHOW show; /* Debug fs function hookup. */
> + struct dentry *dir_entry; /* Debug fs directory entry. */
> + void *priv_data; /* Private data */
> +};
> +
> +/* Global state information. */
> +struct sm_state_t {
> + struct platform_device *pdev;
> +
> + struct miscdevice misc_dev;
> +
> + struct sm_instance *sm_handle; /* Handle for videocore
> service. */
> +
> + spinlock_t kernelid_map_lock; /* Spinlock protecting
> kernelid_map */
> + struct idr kernelid_map;
> +
> + struct mutex map_lock; /* Global map lock. */
> + struct list_head buffer_list; /* List of buffer. */
> +
> + struct vc_sm_privdata_t *data_knl; /* Kernel internal data
> tracking. */
> + struct vc_sm_privdata_t *vpu_allocs; /* All allocations from
> the VPU */
> + struct dentry *dir_root; /* Debug fs entries root. */
> + struct sm_pde_t dir_state; /* Debug fs entries state
> sub-tree. */
> +
> + bool require_released_callback; /* VPU will send a released
> msg when it
> + * has finished with a
> resource.
> + */
> + u32 int_trans_id; /* Interrupted transaction. */
> +};
> +
> +struct vc_sm_dma_buf_attachment {
> + struct device *dev;
> + struct sg_table sg_table;
> + struct list_head list;
> + enum dma_data_direction dma_dir;
> +};
> +
> +/* ---- Private Variables ------------------------------------------
> ----- */
> +
> +static struct sm_state_t *sm_state;
> +static int sm_inited;
> +
> +/* ---- Private Function Prototypes --------------------------------
> ------ */
> +
> +/* ---- Private Functions ------------------------------------------
> ------ */
> +
> +static int get_kernel_id(struct vc_sm_buffer *buffer)
> +{
> + int handle;
> +
> + spin_lock(&sm_state->kernelid_map_lock);
> + handle = idr_alloc(&sm_state->kernelid_map, buffer, 0, 0,
> GFP_KERNEL);
> + spin_unlock(&sm_state->kernelid_map_lock);
> +
> + return handle;
> +}
> +
> +static struct vc_sm_buffer *lookup_kernel_id(int handle)
> +{
> + return idr_find(&sm_state->kernelid_map, handle);
> +}
> +
> +static void free_kernel_id(int handle)
> +{
> + spin_lock(&sm_state->kernelid_map_lock);
> + idr_remove(&sm_state->kernelid_map, handle);
> + spin_unlock(&sm_state->kernelid_map_lock);
> +}
> +
> +static int vc_sm_cma_seq_file_show(struct seq_file *s, void *v)
> +{
> + struct sm_pde_t *sm_pde;
> +
> + sm_pde = (struct sm_pde_t *)(s->private);
> +
> + if (sm_pde && sm_pde->show)
> + sm_pde->show(s, v);
> +
> + return 0;
> +}
> +
> +static int vc_sm_cma_single_open(struct inode *inode, struct file
> *file)
> +{
> + return single_open(file, vc_sm_cma_seq_file_show, inode-
> >i_private);
> +}
> +
> +static const struct file_operations vc_sm_cma_debug_fs_fops = {
> + .open = vc_sm_cma_single_open,
> + .read = seq_read,
> + .llseek = seq_lseek,
> + .release = single_release,
> +};
> +
> +static int vc_sm_cma_global_state_show(struct seq_file *s, void *v)
> +{
> + struct vc_sm_buffer *resource = NULL;
> + int resource_count = 0;
> +
> + if (!sm_state)
> + return 0;
> +
> + seq_printf(s, "\nVC-ServiceHandle %p\n", sm_state-
> >sm_handle);
> +
> + /* Log all applicable mapping(s). */
> +
> + mutex_lock(&sm_state->map_lock);
> + seq_puts(s, "\nResources\n");
> + if (!list_empty(&sm_state->buffer_list)) {
> + list_for_each_entry(resource, &sm_state->buffer_list,
> + global_buffer_list) {
> + resource_count++;
> +
> + seq_printf(s, "\nResource %p\n",
> + resource);
> + seq_printf(s, " NAME %s\n",
> + resource->name);
> + seq_printf(s, " SIZE %zu\n",
> + resource->size);
> + seq_printf(s, " DMABUF %p\n",
> + resource->dma_buf);
> + if (resource->imported) {
> + seq_printf(s,
> " ATTACH %p\n",
> + resource->import.attach);
> + seq_printf(s,
> " SGT %p\n",
> + resource->import.sgt);
> + } else {
> + seq_printf(s,
> " SGT %p\n",
> + resource->alloc.sg_table);
> + }
> + seq_printf(s, " DMA_ADDR %pad\n",
> + &resource->dma_addr);
> + seq_printf(s,
> " VC_HANDLE %08x\n",
> + resource->vc_handle);
> + seq_printf(s, " VC_MAPPING %d\n",
> + resource->vpu_state);
> + }
> + }
> + seq_printf(s, "\n\nTotal resource count: %d\n\n",
> resource_count);
> +
> + mutex_unlock(&sm_state->map_lock);
> +
> + return 0;
> +}
> +
> +/*
> + * Adds a buffer to the private data list which tracks all the
> allocated
> + * data.
> + */
> +static void vc_sm_add_resource(struct vc_sm_privdata_t *privdata,
> + struct vc_sm_buffer *buffer)
> +{
> + mutex_lock(&sm_state->map_lock);
> + list_add(&buffer->global_buffer_list, &sm_state->buffer_list);
> + mutex_unlock(&sm_state->map_lock);
> +
> + pr_debug("[%s]: added buffer %p (name %s, size %zu)\n",
> + __func__, buffer, buffer->name, buffer->size);
> +}
> +
> +/*
> + * Cleans up imported dmabuf.
> + */
> +static void vc_sm_clean_up_dmabuf(struct vc_sm_buffer *buffer)
> +{
> + if (!buffer->imported)
> + return;
> +
> + /* Handle cleaning up imported dmabufs */
> + mutex_lock(&buffer->lock);
> + if (buffer->import.sgt) {
> + dma_buf_unmap_attachment(buffer->import.attach,
> + buffer->import.sgt,
> + DMA_BIDIRECTIONAL);
> + buffer->import.sgt = NULL;
> + }
> + if (buffer->import.attach) {
> + dma_buf_detach(buffer->dma_buf, buffer->import.attach);
> + buffer->import.attach = NULL;
> + }
> + mutex_unlock(&buffer->lock);
> +}
> +
> +/*
> + * Instructs VPU to decrement the refcount on a buffer.
> + */
> +static void vc_sm_vpu_free(struct vc_sm_buffer *buffer)
> +{
> + if (buffer->vc_handle && buffer->vpu_state == VPU_MAPPED) {
> + struct vc_sm_free_t free = { buffer->vc_handle, 0 };
> + int status = vc_sm_cma_vchi_free(sm_state->sm_handle,
> &free,
> + &sm_state->int_trans_id);
> + if (status != 0 && status != -EINTR) {
> + pr_err("[%s]: failed to free memory on
> videocore (status: %u, trans_id: %u)\n",
> + __func__, status, sm_state-
> >int_trans_id);
> + }
> +
> + if (sm_state->require_released_callback) {
> + /* Need to wait for the VPU to confirm the
> free. */
> +
> + /* Retain a reference on this until the VPU has
> + * released it
> + */
> + buffer->vpu_state = VPU_UNMAPPING;
> + } else {
> + buffer->vpu_state = VPU_NOT_MAPPED;
> + buffer->vc_handle = 0;
> + }
> + }
> +}
> +
> +/*
> + * Release an allocation.
> + * All refcounting is done via the dma buf object.
> + *
> + * Must be called with the mutex held. The function will either
> release the
> + * mutex (if defering the release) or destroy it. The caller must
> therefore not
> + * reuse the buffer on return.
> + */
> +static void vc_sm_release_resource(struct vc_sm_buffer *buffer)
> +{
> + pr_debug("[%s]: buffer %p (name %s, size %zu), imported %u\n",
> + __func__, buffer, buffer->name, buffer->size,
> + buffer->imported);
> +
> + if (buffer->vc_handle) {
> + /* We've sent the unmap request but not had the
> response. */
> + pr_debug("[%s]: Waiting for VPU unmap response on
> %p\n",
> + __func__, buffer);
> + goto defer;
> + }
> + if (buffer->in_use) {
> + /* dmabuf still in use - we await the release */
> + pr_debug("[%s]: buffer %p is still in use\n", __func__,
> buffer);
> + goto defer;
> + }
> +
> + /* Release the allocation (whether imported dmabuf or CMA
> allocation) */
> + if (buffer->imported) {
> + if (buffer->import.dma_buf)
> + dma_buf_put(buffer->import.dma_buf);
> + else
> + pr_err("%s: Imported dmabuf already been put
> for buf %p\n",
> + __func__, buffer);
> + buffer->import.dma_buf = NULL;
> + } else {
> + dma_free_coherent(&sm_state->pdev->dev, buffer->size,
> + buffer->cookie, buffer->dma_addr);
> + }
> +
> + /* Free our buffer. Start by removing it from the list */
> + mutex_lock(&sm_state->map_lock);
> + list_del(&buffer->global_buffer_list);
> + mutex_unlock(&sm_state->map_lock);
> +
> + pr_debug("%s: Release our allocation - done\n", __func__);
> + mutex_unlock(&buffer->lock);
> +
> + mutex_destroy(&buffer->lock);
> +
> + kfree(buffer);
> + return;
> +
> +defer:
> + mutex_unlock(&buffer->lock);
> +}
> +
> +/* Create support for private data tracking. */
> +static struct vc_sm_privdata_t *vc_sm_cma_create_priv_data(pid_t id)
> +{
> + char alloc_name[32];
> + struct vc_sm_privdata_t *file_data = NULL;
> +
> + /* Allocate private structure. */
> + file_data = kzalloc(sizeof(*file_data), GFP_KERNEL);
> +
> + if (!file_data)
> + return NULL;
> +
> + snprintf(alloc_name, sizeof(alloc_name), "%d", id);
> +
> + file_data->pid = id;
> +
> + return file_data;
> +}
> +
> +/* Dma buf operations for use with our own allocations */
> +
> +static int vc_sm_dma_buf_attach(struct dma_buf *dmabuf,
> + struct dma_buf_attachment *attachment)
> +
> +{
> + struct vc_sm_dma_buf_attachment *a;
> + struct sg_table *sgt;
> + struct vc_sm_buffer *buf = dmabuf->priv;
> + struct scatterlist *rd, *wr;
> + int ret, i;
> +
> + a = kzalloc(sizeof(*a), GFP_KERNEL);
> + if (!a)
> + return -ENOMEM;
> +
> + pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf,
> attachment);
> +
> + mutex_lock(&buf->lock);
> +
> + INIT_LIST_HEAD(&a->list);
> +
> + sgt = &a->sg_table;
> +
> + /* Copy the buf->base_sgt scatter list to the attachment, as we
> can't
> + * map the same scatter list to multiple attachments at the
> same time.
> + */
> + ret = sg_alloc_table(sgt, buf->alloc.sg_table->orig_nents,
> GFP_KERNEL);
> + if (ret) {
> + kfree(a);
> + return -ENOMEM;
> + }
> +
> + rd = buf->alloc.sg_table->sgl;
> + wr = sgt->sgl;
> + for (i = 0; i < sgt->orig_nents; ++i) {
> + sg_set_page(wr, sg_page(rd), rd->length, rd->offset);
> + rd = sg_next(rd);
> + wr = sg_next(wr);
> + }
> +
> + a->dma_dir = DMA_NONE;
> + attachment->priv = a;
> +
> + list_add(&a->list, &buf->attachments);
> + mutex_unlock(&buf->lock);
> +
> + return 0;
> +}
> +
> +static void vc_sm_dma_buf_detach(struct dma_buf *dmabuf,
> + struct dma_buf_attachment *attachment)
> +{
> + struct vc_sm_dma_buf_attachment *a = attachment->priv;
> + struct vc_sm_buffer *buf = dmabuf->priv;
> + struct sg_table *sgt;
> +
> + pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf,
> attachment);
> + if (!a)
> + return;
> +
> + sgt = &a->sg_table;
> +
> + /* release the scatterlist cache */
> + if (a->dma_dir != DMA_NONE)
> + dma_unmap_sg(attachment->dev, sgt->sgl, sgt-
> >orig_nents,
> + a->dma_dir);
> + sg_free_table(sgt);
> +
> + mutex_lock(&buf->lock);
> + list_del(&a->list);
> + mutex_unlock(&buf->lock);
> +
> + kfree(a);
> +}
> +
> +static struct sg_table *vc_sm_map_dma_buf(struct dma_buf_attachment
> *attachment,
> + enum dma_data_direction
> direction)
> +{
> + struct vc_sm_dma_buf_attachment *a = attachment->priv;
> + /* stealing dmabuf mutex to serialize map/unmap operations */
> + struct mutex *lock = &attachment->dmabuf->lock;
> + struct sg_table *table;
> +
> + mutex_lock(lock);
> + pr_debug("%s attachment %p\n", __func__, attachment);
> + table = &a->sg_table;
> +
> + /* return previously mapped sg table */
> + if (a->dma_dir == direction) {
> + mutex_unlock(lock);
> + return table;
> + }
> +
> + /* release any previous cache */
> + if (a->dma_dir != DMA_NONE) {
> + dma_unmap_sg(attachment->dev, table->sgl, table-
> >orig_nents,
> + a->dma_dir);
> + a->dma_dir = DMA_NONE;
> + }
> +
> + /* mapping to the client with new direction */
> + table->nents = dma_map_sg(attachment->dev, table->sgl,
> + table->orig_nents, direction);
> + if (!table->nents) {
> + pr_err("failed to map scatterlist\n");
> + mutex_unlock(lock);
> + return ERR_PTR(-EIO);
> + }
> +
> + a->dma_dir = direction;
> + mutex_unlock(lock);
> +
> + pr_debug("%s attachment %p\n", __func__, attachment);
> + return table;
> +}
> +
> +static void vc_sm_unmap_dma_buf(struct dma_buf_attachment
> *attachment,
> + struct sg_table *table,
> + enum dma_data_direction direction)
> +{
> + pr_debug("%s attachment %p\n", __func__, attachment);
> + dma_unmap_sg(attachment->dev, table->sgl, table->nents,
> direction);
> +}
> +
> +static int vc_sm_dmabuf_mmap(struct dma_buf *dmabuf, struct
> vm_area_struct *vma)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> + int ret;
> +
> + pr_debug("%s dmabuf %p, buf %p, vm_start %08lX\n", __func__,
> dmabuf,
> + buf, vma->vm_start);
> +
> + mutex_lock(&buf->lock);
> +
> + /* now map it to userspace */
> + vma->vm_pgoff = 0;
> +
> + ret = dma_mmap_coherent(&sm_state->pdev->dev, vma, buf->cookie,
> + buf->dma_addr, buf->size);
> +
> + if (ret) {
> + pr_err("Remapping memory failed, error: %d\n", ret);
> + return ret;
> + }
> +
> + vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
> +
> + mutex_unlock(&buf->lock);
> +
> + if (ret)
> + pr_err("%s: failure mapping buffer to userspace\n",
> + __func__);
> +
> + return ret;
> +}
> +
> +static void vc_sm_dma_buf_release(struct dma_buf *dmabuf)
> +{
> + struct vc_sm_buffer *buffer;
> +
> + if (!dmabuf)
> + return;
> +
> + buffer = (struct vc_sm_buffer *)dmabuf->priv;
> +
> + mutex_lock(&buffer->lock);
> +
> + pr_debug("%s dmabuf %p, buffer %p\n", __func__, dmabuf,
> buffer);
> +
> + buffer->in_use = 0;
> +
> + /* Unmap on the VPU */
> + vc_sm_vpu_free(buffer);
> + pr_debug("%s vpu_free done\n", __func__);
> +
> + /* Unmap our dma_buf object (the vc_sm_buffer remains until
> released
> + * on the VPU).
> + */
> + vc_sm_clean_up_dmabuf(buffer);
> + pr_debug("%s clean_up dmabuf done\n", __func__);
> +
> + /* buffer->lock will be destroyed by vc_sm_release_resource if
> finished
> + * with, otherwise unlocked. Do NOT unlock here.
> + */
> + vc_sm_release_resource(buffer);
> + pr_debug("%s done\n", __func__);
> +}
> +
> +static int vc_sm_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction
> direction)
> +{
> + struct vc_sm_buffer *buf;
> + struct vc_sm_dma_buf_attachment *a;
> +
> + if (!dmabuf)
> + return -EFAULT;
> +
> + buf = dmabuf->priv;
> + if (!buf)
> + return -EFAULT;
> +
> + mutex_lock(&buf->lock);
> +
> + list_for_each_entry(a, &buf->attachments, list) {
> + dma_sync_sg_for_cpu(a->dev, a->sg_table.sgl,
> + a->sg_table.nents, direction);
> + }
> + mutex_unlock(&buf->lock);
> +
> + return 0;
> +}
> +
> +static int vc_sm_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction
> direction)
> +{
> + struct vc_sm_buffer *buf;
> + struct vc_sm_dma_buf_attachment *a;
> +
> + if (!dmabuf)
> + return -EFAULT;
> + buf = dmabuf->priv;
> + if (!buf)
> + return -EFAULT;
> +
> + mutex_lock(&buf->lock);
> +
> + list_for_each_entry(a, &buf->attachments, list) {
> + dma_sync_sg_for_device(a->dev, a->sg_table.sgl,
> + a->sg_table.nents, direction);
> + }
> + mutex_unlock(&buf->lock);
> +
> + return 0;
> +}
> +
> +static const struct dma_buf_ops dma_buf_ops = {
> + .map_dma_buf = vc_sm_map_dma_buf,
> + .unmap_dma_buf = vc_sm_unmap_dma_buf,
> + .mmap = vc_sm_dmabuf_mmap,
> + .release = vc_sm_dma_buf_release,
> + .attach = vc_sm_dma_buf_attach,
> + .detach = vc_sm_dma_buf_detach,
> + .begin_cpu_access = vc_sm_dma_buf_begin_cpu_access,
> + .end_cpu_access = vc_sm_dma_buf_end_cpu_access,
> +};
> +
> +/* Dma_buf operations for chaining through to an imported dma_buf */
> +
> +static
> +int vc_sm_import_dma_buf_attach(struct dma_buf *dmabuf,
> + struct dma_buf_attachment *attachment)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + if (!buf->imported)
> + return -EINVAL;
> + return buf->import.dma_buf->ops->attach(buf->import.dma_buf,
> + attachment);
> +}
> +
> +static
> +void vc_sm_import_dma_buf_detatch(struct dma_buf *dmabuf,
> + struct dma_buf_attachment
> *attachment)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + if (!buf->imported)
> + return;
> + buf->import.dma_buf->ops->detach(buf->import.dma_buf,
> attachment);
> +}
> +
> +static
> +struct sg_table *vc_sm_import_map_dma_buf(struct dma_buf_attachment
> *attachment,
> + enum dma_data_direction
> direction)
> +{
> + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> +
> + if (!buf->imported)
> + return NULL;
> + return buf->import.dma_buf->ops->map_dma_buf(attachment,
> + direction);
> +}
> +
> +static
> +void vc_sm_import_unmap_dma_buf(struct dma_buf_attachment
> *attachment,
> + struct sg_table *table,
> + enum dma_data_direction direction)
> +{
> + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> +
> + if (!buf->imported)
> + return;
> + buf->import.dma_buf->ops->unmap_dma_buf(attachment, table,
> direction);
> +}
> +
> +static
> +int vc_sm_import_dmabuf_mmap(struct dma_buf *dmabuf, struct
> vm_area_struct *vma)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + pr_debug("%s: mmap dma_buf %p, buf %p, imported db %p\n",
> __func__,
> + dmabuf, buf, buf->import.dma_buf);
> + if (!buf->imported) {
> + pr_err("%s: mmap dma_buf %p- not an imported buffer\n",
> + __func__, dmabuf);
> + return -EINVAL;
> + }
> + return buf->import.dma_buf->ops->mmap(buf->import.dma_buf,
> vma);
> +}
> +
> +static
> +void vc_sm_import_dma_buf_release(struct dma_buf *dmabuf)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + pr_debug("%s: Relasing dma_buf %p\n", __func__, dmabuf);
> + mutex_lock(&buf->lock);
> + if (!buf->imported)
> + return;
> +
> + buf->in_use = 0;
> +
> + vc_sm_vpu_free(buf);
> +
> + vc_sm_release_resource(buf);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction
> direction)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + if (!buf->imported)
> + return -EINVAL;
> + return buf->import.dma_buf->ops->begin_cpu_access(buf-
> >import.dma_buf,
> + direction);
> +}
> +
> +static
> +int vc_sm_import_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> + enum dma_data_direction
> direction)
> +{
> + struct vc_sm_buffer *buf = dmabuf->priv;
> +
> + if (!buf->imported)
> + return -EINVAL;
> + return buf->import.dma_buf->ops->end_cpu_access(buf-
> >import.dma_buf,
> + direction);
> +}
> +
> +static const struct dma_buf_ops dma_buf_import_ops = {
> + .map_dma_buf = vc_sm_import_map_dma_buf,
> + .unmap_dma_buf = vc_sm_import_unmap_dma_buf,
> + .mmap = vc_sm_import_dmabuf_mmap,
> + .release = vc_sm_import_dma_buf_release,
> + .attach = vc_sm_import_dma_buf_attach,
> + .detach = vc_sm_import_dma_buf_detatch,
> + .begin_cpu_access = vc_sm_import_dma_buf_begin_cpu_access,
> + .end_cpu_access = vc_sm_import_dma_buf_end_cpu_access,
> +};
> +
> +/* Import a dma_buf to be shared with VC. */
> +int
> +vc_sm_cma_import_dmabuf_internal(struct vc_sm_privdata_t *private,
> + struct dma_buf *dma_buf,
> + int fd,
> + struct dma_buf **imported_buf)
> +{
> + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> + struct vc_sm_buffer *buffer = NULL;
> + struct vc_sm_import import = { };
> + struct vc_sm_import_result result = { };
> + struct dma_buf_attachment *attach = NULL;
> + struct sg_table *sgt = NULL;
> + dma_addr_t dma_addr;
> + int ret = 0;
> + int status;
> +
> + /* Setup our allocation parameters */
> + pr_debug("%s: importing dma_buf %p/fd %d\n", __func__, dma_buf,
> fd);
> +
> + if (fd < 0)
> + get_dma_buf(dma_buf);
> + else
> + dma_buf = dma_buf_get(fd);
> +
> + if (!dma_buf)
> + return -EINVAL;
> +
> + attach = dma_buf_attach(dma_buf, &sm_state->pdev->dev);
> + if (IS_ERR(attach)) {
> + ret = PTR_ERR(attach);
> + goto error;
> + }
> +
> + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> + if (IS_ERR(sgt)) {
> + ret = PTR_ERR(sgt);
> + goto error;
> + }
> +
> + /* Verify that the address block is contiguous */
> + if (sgt->nents != 1) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + /* Allocate local buffer to track this allocation. */
> + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> + if (!buffer) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + import.type = VC_SM_ALLOC_NON_CACHED;
> + dma_addr = sg_dma_address(sgt->sgl);
> + import.addr = (u32)dma_addr;
> + if ((import.addr & 0xC0000000) != 0xC0000000) {
> + pr_err("%s: Expecting an uncached alias for dma_addr
> %pad\n",
> + __func__, &dma_addr);
> + import.addr |= 0xC0000000;
> + }
> + import.size = sg_dma_len(sgt->sgl);
> + import.allocator = current->tgid;
> + import.kernel_id = get_kernel_id(buffer);
> +
> + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> +
> + pr_debug("[%s]: attempt to import \"%s\" data - type %u, addr
> %pad, size %u.\n",
> + __func__, import.name, import.type, &dma_addr,
> import.size);
> +
> + /* Allocate the videocore buffer. */
> + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import,
> &result,
> + &sm_state->int_trans_id);
> + if (status == -EINTR) {
> + pr_debug("[%s]: requesting import memory action restart
> (trans_id: %u)\n",
> + __func__, sm_state->int_trans_id);
> + ret = -ERESTARTSYS;
> + private->restart_sys = -EINTR;
> + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> + goto error;
> + } else if (status || !result.res_handle) {
> + pr_debug("[%s]: failed to import memory on videocore
> (status: %u, trans_id: %u)\n",
> + __func__, status, sm_state->int_trans_id);
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + mutex_init(&buffer->lock);
> + INIT_LIST_HEAD(&buffer->attachments);
> + memcpy(buffer->name, import.name,
> + min(sizeof(buffer->name), sizeof(import.name) - 1));
> +
> + /* Keep track of the buffer we created. */
> + buffer->private = private;
> + buffer->vc_handle = result.res_handle;
> + buffer->size = import.size;
> + buffer->vpu_state = VPU_MAPPED;
> +
> + buffer->imported = 1;
> + buffer->import.dma_buf = dma_buf;
> +
> + buffer->import.attach = attach;
> + buffer->import.sgt = sgt;
> + buffer->dma_addr = dma_addr;
> + buffer->in_use = 1;
> + buffer->kernel_id = import.kernel_id;
> +
> + /*
> + * We're done - we need to export a new dmabuf chaining through
> most
> + * functions, but enabling us to release our own internal
> references
> + * here.
> + */
> + exp_info.ops = &dma_buf_import_ops;
> + exp_info.size = import.size;
> + exp_info.flags = O_RDWR;
> + exp_info.priv = buffer;
> +
> + buffer->dma_buf = dma_buf_export(&exp_info);
> + if (IS_ERR(buffer->dma_buf)) {
> + ret = PTR_ERR(buffer->dma_buf);
> + goto error;
> + }
> +
> + vc_sm_add_resource(private, buffer);
> +
> + *imported_buf = buffer->dma_buf;
> +
> + return 0;
> +
> +error:
> + if (result.res_handle) {
> + struct vc_sm_free_t free = { result.res_handle, 0 };
> +
> + vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
> + &sm_state->int_trans_id);
> + }
> + free_kernel_id(import.kernel_id);
> + kfree(buffer);
> + if (sgt)
> + dma_buf_unmap_attachment(attach, sgt,
> DMA_BIDIRECTIONAL);
> + if (attach)
> + dma_buf_detach(dma_buf, attach);
> + dma_buf_put(dma_buf);
> + return ret;
> +}
> +
> +static int vc_sm_cma_vpu_alloc(u32 size, u32 align, const char
> *name,
> + u32 mem_handle, struct vc_sm_buffer
> **ret_buffer)
> +{
> + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> + struct vc_sm_buffer *buffer = NULL;
> + struct sg_table *sgt;
> + int aligned_size;
> + int ret = 0;
> +
> + /* Align to the user requested align */
> + aligned_size = ALIGN(size, align);
> + /* and then to a page boundary */
> + aligned_size = PAGE_ALIGN(aligned_size);
> +
> + if (!aligned_size)
> + return -EINVAL;
> +
> + /* Allocate local buffer to track this allocation. */
> + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> + if (!buffer)
> + return -ENOMEM;
> +
> + mutex_init(&buffer->lock);
> + /* Acquire the mutex as vc_sm_release_resource will release it
> in the
> + * error path.
> + */
> + mutex_lock(&buffer->lock);
> +
> + buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
> + aligned_size, &buffer-
> >dma_addr,
> + GFP_KERNEL);
> + if (!buffer->cookie) {
> + pr_err("[%s]: dma_alloc_coherent alloc of %d bytes
> failed\n",
> + __func__, aligned_size);
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + pr_debug("[%s]: alloc of %d bytes success\n",
> + __func__, aligned_size);
> +
> + sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
> + if (!sgt) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer-
> >cookie,
> + buffer->dma_addr, buffer->size);
> + if (ret < 0) {
> + pr_err("failed to get scatterlist from DMA API\n");
> + kfree(sgt);
> + ret = -ENOMEM;
> + goto error;
> + }
> + buffer->alloc.sg_table = sgt;
> +
> + INIT_LIST_HEAD(&buffer->attachments);
> +
> + memcpy(buffer->name, name,
> + min(sizeof(buffer->name), strlen(name)));
> +
> + exp_info.ops = &dma_buf_ops;
> + exp_info.size = aligned_size;
> + exp_info.flags = O_RDWR;
> + exp_info.priv = buffer;
> +
> + buffer->dma_buf = dma_buf_export(&exp_info);
> + if (IS_ERR(buffer->dma_buf)) {
> + ret = PTR_ERR(buffer->dma_buf);
> + goto error;
> + }
> + buffer->dma_addr = (u32)sg_dma_address(buffer->alloc.sg_table-
> >sgl);
> + if ((buffer->dma_addr & 0xC0000000) != 0xC0000000) {
> + pr_warn_once("%s: Expecting an uncached alias for
> dma_addr %pad\n",
> + __func__, &buffer->dma_addr);
> + buffer->dma_addr |= 0xC0000000;
> + }
> + buffer->private = sm_state->vpu_allocs;
> +
> + buffer->vc_handle = mem_handle;
> + buffer->vpu_state = VPU_MAPPED;
> + buffer->vpu_allocated = 1;
> + buffer->size = size;
> + /*
> + * Create an ID that will be passed along with our message so
> + * that when we service the release reply, we can look up which
> + * resource is being released.
> + */
> + buffer->kernel_id = get_kernel_id(buffer);
> +
> + vc_sm_add_resource(sm_state->vpu_allocs, buffer);
> +
> + mutex_unlock(&buffer->lock);
> +
> + *ret_buffer = buffer;
> + return 0;
> +error:
> + if (buffer)
> + vc_sm_release_resource(buffer);
> + return ret;
> +}
> +
> +static void
> +vc_sm_vpu_event(struct sm_instance *instance, struct vc_sm_result_t
> *reply,
> + int reply_len)
> +{
> + switch (reply->trans_id & ~0x80000000) {
> + case VC_SM_MSG_TYPE_CLIENT_VERSION:
> + {
> + /* Acknowledge that the firmware supports the version
> command */
> + pr_debug("%s: firmware acked version msg. Require
> release cb\n",
> + __func__);
> + sm_state->require_released_callback = true;
> + }
> + break;
> + case VC_SM_MSG_TYPE_RELEASED:
> + {
> + struct vc_sm_released *release = (struct vc_sm_released
> *)reply;
> + struct vc_sm_buffer *buffer =
> + lookup_kernel_id(release-
> >kernel_id);
> + if (!buffer) {
> + pr_err("%s: VC released a buffer that is
> already released, kernel_id %d\n",
> + __func__, release->kernel_id);
> + break;
> + }
> + mutex_lock(&buffer->lock);
> +
> + pr_debug("%s: Released addr %08x, size %u, id %08x,
> mem_handle %08x\n",
> + __func__, release->addr, release->size,
> + release->kernel_id, release->vc_handle);
> +
> + buffer->vc_handle = 0;
> + buffer->vpu_state = VPU_NOT_MAPPED;
> + free_kernel_id(release->kernel_id);
> +
> + if (buffer->vpu_allocated) {
> + /* VPU allocation, so release the dmabuf which
> will
> + * trigger the clean up.
> + */
> + mutex_unlock(&buffer->lock);
> + dma_buf_put(buffer->dma_buf);
> + } else {
> + vc_sm_release_resource(buffer);
> + }
> + }
> + break;
> + case VC_SM_MSG_TYPE_VC_MEM_REQUEST:
> + {
> + struct vc_sm_buffer *buffer = NULL;
> + struct vc_sm_vc_mem_request *req =
> + (struct vc_sm_vc_mem_request
> *)reply;
> + struct vc_sm_vc_mem_request_result reply;
> + int ret;
> +
> + pr_debug("%s: Request %u bytes of memory, align %d name
> %s, trans_id %08x\n",
> + __func__, req->size, req->align, req->name,
> + req->trans_id);
> + ret = vc_sm_cma_vpu_alloc(req->size, req->align, req-
> >name,
> + req->vc_handle, &buffer);
> +
> + reply.trans_id = req->trans_id;
> + if (!ret) {
> + reply.addr = buffer->dma_addr;
> + reply.kernel_id = buffer->kernel_id;
> + pr_debug("%s: Allocated resource buffer %p,
> addr %pad\n",
> + __func__, buffer, &buffer->dma_addr);
> + } else {
> + pr_err("%s: Allocation failed size %u, name %s,
> vc_handle %u\n",
> + __func__, req->size, req->name, req-
> >vc_handle);
> + reply.addr = 0;
> + reply.kernel_id = 0;
> + }
> + vc_sm_vchi_client_vc_mem_req_reply(sm_state->sm_handle,
> &reply,
> + &sm_state-
> >int_trans_id);
> + break;
> + }
> + break;
> + default:
> + pr_err("%s: Unknown vpu cmd %x\n", __func__, reply-
> >trans_id);
> + break;
> + }
> +}
> +
> +/* Userspace handling */
> +/*
> + * Open the device. Creates a private state to help track all
> allocation
> + * associated with this device.
> + */
> +static int vc_sm_cma_open(struct inode *inode, struct file *file)
> +{
> + /* Make sure the device was started properly. */
> + if (!sm_state) {
> + pr_err("[%s]: invalid device\n", __func__);
> + return -EPERM;
> + }
> +
> + file->private_data = vc_sm_cma_create_priv_data(current->tgid);
> + if (!file->private_data) {
> + pr_err("[%s]: failed to create data tracker\n",
> __func__);
> +
> + return -ENOMEM;
> + }
> +
> + return 0;
> +}
> +
> +/*
> + * Close the vcsm-cma device.
> + * All allocations are file descriptors to the dmabuf objects, so we
> will get
> + * the clean up request on those as those are cleaned up.
> + */
> +static int vc_sm_cma_release(struct inode *inode, struct file *file)
> +{
> + struct vc_sm_privdata_t *file_data =
> + (struct vc_sm_privdata_t *)file->private_data;
> + int ret = 0;
> +
> + /* Make sure the device was started properly. */
> + if (!sm_state || !file_data) {
> + pr_err("[%s]: invalid device\n", __func__);
> + ret = -EPERM;
> + goto out;
> + }
> +
> + pr_debug("[%s]: using private data %p\n", __func__, file_data);
> +
> + /* Terminate the private data. */
> + kfree(file_data);
> +
> +out:
> + return ret;
> +}
> +
> +/*
> + * Allocate a shared memory handle and block.
> + * Allocation is from CMA, and then imported into the VPU mappings.
> + */
> +int vc_sm_cma_ioctl_alloc(struct vc_sm_privdata_t *private,
> + struct vc_sm_cma_ioctl_alloc *ioparam)
> +{
> + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> + struct vc_sm_buffer *buffer = NULL;
> + struct vc_sm_import import = { 0 };
> + struct vc_sm_import_result result = { 0 };
> + struct dma_buf *dmabuf = NULL;
> + struct sg_table *sgt;
> + int aligned_size;
> + int ret = 0;
> + int status;
> + int fd = -1;
> +
> + aligned_size = PAGE_ALIGN(ioparam->size);
> +
> + if (!aligned_size)
> + return -EINVAL;
> +
> + /* Allocate local buffer to track this allocation. */
> + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> + if (!buffer) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
> + aligned_size,
> + &buffer->dma_addr,
> + GFP_KERNEL);
> + if (!buffer->cookie) {
> + pr_err("[%s]: dma_alloc_coherent alloc of %d bytes
> failed\n",
> + __func__, aligned_size);
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + import.type = VC_SM_ALLOC_NON_CACHED;
> + import.allocator = current->tgid;
> +
> + if (*ioparam->name)
> + memcpy(import.name, ioparam->name, sizeof(import.name)
> - 1);
> + else
> + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> +
> + mutex_init(&buffer->lock);
> + INIT_LIST_HEAD(&buffer->attachments);
> + memcpy(buffer->name, import.name,
> + min(sizeof(buffer->name), sizeof(import.name) - 1));
> +
> + exp_info.ops = &dma_buf_ops;
> + exp_info.size = aligned_size;
> + exp_info.flags = O_RDWR;
> + exp_info.priv = buffer;
> +
> + dmabuf = dma_buf_export(&exp_info);
> + if (IS_ERR(dmabuf)) {
> + ret = PTR_ERR(dmabuf);
> + goto error;
> + }
> + buffer->dma_buf = dmabuf;
> +
> + import.addr = buffer->dma_addr;
> + import.size = aligned_size;
> + import.kernel_id = get_kernel_id(buffer);
> +
> + /* Wrap it into a videocore buffer. */
> + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import,
> &result,
> + &sm_state->int_trans_id);
> + if (status == -EINTR) {
> + pr_debug("[%s]: requesting import memory action restart
> (trans_id: %u)\n",
> + __func__, sm_state->int_trans_id);
> + ret = -ERESTARTSYS;
> + private->restart_sys = -EINTR;
> + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> + goto error;
> + } else if (status || !result.res_handle) {
> + pr_err("[%s]: failed to import memory on videocore
> (status: %u, trans_id: %u)\n",
> + __func__, status, sm_state->int_trans_id);
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + /* Keep track of the buffer we created. */
> + buffer->private = private;
> + buffer->vc_handle = result.res_handle;
> + buffer->size = import.size;
> + buffer->vpu_state = VPU_MAPPED;
> + buffer->kernel_id = import.kernel_id;
> +
> + sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
> + if (!sgt) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer-
> >cookie,
> + buffer->dma_addr, buffer->size);
> + if (ret < 0) {
> + /* FIXME: error handling */
> + pr_err("failed to get scatterlist from DMA API\n");
> + kfree(sgt);
> + ret = -ENOMEM;
> + goto error;
> + }
> + buffer->alloc.sg_table = sgt;
> +
> + fd = dma_buf_fd(dmabuf, O_CLOEXEC);
> + if (fd < 0)
> + goto error;
> +
> + vc_sm_add_resource(private, buffer);
> +
> + pr_debug("[%s]: Added resource as fd %d, buffer %p, private %p,
> dma_addr %pad\n",
> + __func__, fd, buffer, private, &buffer->dma_addr);
> +
> + /* We're done */
> + ioparam->handle = fd;
> + ioparam->vc_handle = buffer->vc_handle;
> + ioparam->dma_addr = buffer->dma_addr;
> + return 0;
> +
> +error:
> + pr_err("[%s]: something failed - cleanup. ret %d\n", __func__,
> ret);
> +
> + if (dmabuf) {
> + /* dmabuf has been exported, therefore allow dmabuf
> cleanup to
> + * deal with this
> + */
> + dma_buf_put(dmabuf);
> + } else {
> + /* No dmabuf, therefore just free the buffer here */
> + if (buffer->cookie)
> + dma_free_coherent(&sm_state->pdev->dev, buffer-
> >size,
> + buffer->cookie, buffer-
> >dma_addr);
> + kfree(buffer);
> + }
> + return ret;
> +}
> +
> +#ifndef CONFIG_ARM64
> +/* Converts VCSM_CACHE_OP_* to an operating function. */
> +static void (*cache_op_to_func(const unsigned int cache_op))
> + (const void*, const
> void*)
> +{
> + switch (cache_op) {
> + case VC_SM_CACHE_OP_NOP:
> + return NULL;
> +
> + case VC_SM_CACHE_OP_INV:
> + case VC_SM_CACHE_OP_CLEAN:
> + case VC_SM_CACHE_OP_FLUSH:
> + return dmac_flush_range;
> +
> + default:
> + pr_err("[%s]: Invalid cache_op: 0x%08x\n", __func__,
> cache_op);
> + return NULL;
> + }
> +}
> +
> +/*
> + * Clean/invalid/flush cache of which buffer is already pinned (i.e.
> accessed).
> + */
> +static int clean_invalid_contig_2d(const void __user *addr,
> + const size_t block_count,
> + const size_t block_size,
> + const size_t stride,
> + const unsigned int cache_op)
> +{
> + size_t i;
> + void (*op_fn)(const void *start, const void *end);
> +
> + if (!block_size) {
> + pr_err("[%s]: size cannot be 0\n", __func__);
> + return -EINVAL;
> + }
> +
> + op_fn = cache_op_to_func(cache_op);
> + if (!op_fn)
> + return -EINVAL;
> +
> + for (i = 0; i < block_count; i ++, addr += stride)
> + op_fn(addr, addr + block_size);
> +
> + return 0;
> +}
> +
> +static int vc_sm_cma_clean_invalid2(unsigned int cmdnr, unsigned
> long arg)
> +{
> + struct vc_sm_cma_ioctl_clean_invalid2 ioparam;
> + struct vc_sm_cma_ioctl_clean_invalid_block *block = NULL;
> + int i, ret = 0;
> +
> + /* Get parameter data. */
> + if (copy_from_user(&ioparam, (void *)arg, sizeof(ioparam))) {
> + pr_err("[%s]: failed to copy-from-user header for cmd
> %x\n",
> + __func__, cmdnr);
> + return -EFAULT;
> + }
> + block = kmalloc(ioparam.op_count * sizeof(*block), GFP_KERNEL);
> + if (!block)
> + return -EFAULT;
> +
> + if (copy_from_user(block, (void *)(arg + sizeof(ioparam)),
> + ioparam.op_count * sizeof(*block)) != 0) {
> + pr_err("[%s]: failed to copy-from-user payload for cmd
> %x\n",
> + __func__, cmdnr);
> + ret = -EFAULT;
> + goto out;
> + }
> +
> + for (i = 0; i < ioparam.op_count; i++) {
> + const struct vc_sm_cma_ioctl_clean_invalid_block *
> const op =
> + block +
> i;
> +
> + if (op->invalidate_mode == VC_SM_CACHE_OP_NOP)
> + continue;
> +
> + ret = clean_invalid_contig_2d((void __user *)op-
> >start_address,
> + op->block_count, op-
> >block_size,
> + op->inter_block_stride,
> + op->invalidate_mode);
> + if (ret)
> + break;
> + }
> +out:
> + kfree(block);
> +
> + return ret;
> +}
> +#endif
> +
> +static long vc_sm_cma_ioctl(struct file *file, unsigned int cmd,
> + unsigned long arg)
> +{
> + int ret = 0;
> + unsigned int cmdnr = _IOC_NR(cmd);
> + struct vc_sm_privdata_t *file_data =
> + (struct vc_sm_privdata_t *)file->private_data;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !file_data) {
> + pr_err("[%s]: invalid device\n", __func__);
> + return -EPERM;
> + }
> +
> + /* Action is a re-post of a previously interrupted action? */
> + if (file_data->restart_sys == -EINTR) {
> + struct vc_sm_action_clean_t action_clean;
> +
> + pr_debug("[%s]: clean up of action %u (trans_id: %u)
> following EINTR\n",
> + __func__, file_data->int_action,
> + file_data->int_trans_id);
> +
> + action_clean.res_action = file_data->int_action;
> + action_clean.action_trans_id = file_data->int_trans_id;
> +
> + file_data->restart_sys = 0;
> + }
> +
> + /* Now process the command. */
> + switch (cmdnr) {
> + /* New memory allocation.
> + */
> + case VC_SM_CMA_CMD_ALLOC:
> + {
> + struct vc_sm_cma_ioctl_alloc ioparam;
> +
> + /* Get the parameter data. */
> + if (copy_from_user
> + (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
> + pr_err("[%s]: failed to copy-from-user for cmd
> %x\n",
> + __func__, cmdnr);
> + ret = -EFAULT;
> + break;
> + }
> +
> + ret = vc_sm_cma_ioctl_alloc(file_data, &ioparam);
> + if (!ret &&
> + (copy_to_user((void *)arg, &ioparam,
> + sizeof(ioparam)) != 0)) {
> + /* FIXME: Release allocation */
> + pr_err("[%s]: failed to copy-to-user for cmd
> %x\n",
> + __func__, cmdnr);
> + ret = -EFAULT;
> + }
> + break;
> + }
> +
> + case VC_SM_CMA_CMD_IMPORT_DMABUF:
> + {
> + struct vc_sm_cma_ioctl_import_dmabuf ioparam;
> + struct dma_buf *new_dmabuf;
> +
> + /* Get the parameter data. */
> + if (copy_from_user
> + (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
> + pr_err("[%s]: failed to copy-from-user for cmd
> %x\n",
> + __func__, cmdnr);
> + ret = -EFAULT;
> + break;
> + }
> +
> + ret = vc_sm_cma_import_dmabuf_internal(file_data,
> + NULL,
> + ioparam.dmabuf_f
> d,
> + &new_dmabuf);
> +
> + if (!ret) {
> + struct vc_sm_buffer *buf = new_dmabuf->priv;
> +
> + ioparam.size = buf->size;
> + ioparam.handle = dma_buf_fd(new_dmabuf,
> + O_CLOEXEC);
> + ioparam.vc_handle = buf->vc_handle;
> + ioparam.dma_addr = buf->dma_addr;
> +
> + if (ioparam.handle < 0 ||
> + (copy_to_user((void *)arg, &ioparam,
> + sizeof(ioparam)) != 0)) {
> + dma_buf_put(new_dmabuf);
> + /* FIXME: Release allocation */
> + ret = -EFAULT;
> + }
> + }
> + break;
> + }
> +
> +#ifndef CONFIG_ARM64
> + /*
> + * Flush/Invalidate the cache for a given mapping.
> + * Blocks must be pinned (i.e. accessed) before this call.
> + */
> + case VC_SM_CMA_CMD_CLEAN_INVALID2:
> + ret = vc_sm_cma_clean_invalid2(cmdnr, arg);
> + break;
> +#endif
> +
> + default:
> + pr_debug("[%s]: cmd %x tgid %u, owner %u\n", __func__,
> cmdnr,
> + current->tgid, file_data->pid);
> +
> + ret = -EINVAL;
> + break;
> + }
> +
> + return ret;
> +}
> +
> +#ifdef CONFIG_COMPAT
> +struct vc_sm_cma_ioctl_clean_invalid2_32 {
> + u32 op_count;
> + struct vc_sm_cma_ioctl_clean_invalid_block_32 {
> + u16 invalidate_mode;
> + u16 block_count;
> + compat_uptr_t start_address;
> + u32 block_size;
> + u32 inter_block_stride;
> + } s[0];
> +};
> +
> +#define VC_SM_CMA_CMD_CLEAN_INVALID2_32\
> + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
> + struct vc_sm_cma_ioctl_clean_invalid2_32)
> +
> +static long vc_sm_cma_compat_ioctl(struct file *file, unsigned int
> cmd,
> + unsigned long arg)
> +{
> + switch (cmd) {
> + case VC_SM_CMA_CMD_CLEAN_INVALID2_32:
> + /* FIXME */
> + return -EINVAL;
> +
> + default:
> + return vc_sm_cma_ioctl(file, cmd, arg);
> + }
> +}
> +#endif
> +
> +/* Device operations that we managed in this driver. */
> +static const struct file_operations vc_sm_ops = {
> + .owner = THIS_MODULE,
> + .unlocked_ioctl = vc_sm_cma_ioctl,
> +#ifdef CONFIG_COMPAT
> + .compat_ioctl = vc_sm_cma_compat_ioctl,
> +#endif
> + .open = vc_sm_cma_open,
> + .release = vc_sm_cma_release,
> +};
> +
> +/* Driver load/unload functions */
> +/* Videocore connected. */
> +static void vc_sm_connected_init(void)
> +{
> + int ret;
> + struct vchi_instance_handle *vchi_instance;
> + struct vc_sm_version version;
> + struct vc_sm_result_t version_result;
> +
> + pr_info("[%s]: start\n", __func__);
> +
> + /*
> + * Initialize and create a VCHI connection for the shared
> memory service
> + * running on videocore.
> + */
> + ret = vchi_initialise(&vchi_instance);
> + if (ret) {
> + pr_err("[%s]: failed to initialise VCHI instance
> (ret=%d)\n",
> + __func__, ret);
> +
> + return;
> + }
> +
> + ret = vchi_connect(vchi_instance);
> + if (ret) {
> + pr_err("[%s]: failed to connect VCHI instance
> (ret=%d)\n",
> + __func__, ret);
> +
> + return;
> + }
> +
> + /* Initialize an instance of the shared memory service. */
> + sm_state->sm_handle = vc_sm_cma_vchi_init(vchi_instance, 1,
> + vc_sm_vpu_event);
> + if (!sm_state->sm_handle) {
> + pr_err("[%s]: failed to initialize shared memory
> service\n",
> + __func__);
> +
> + return;
> + }
> +
> + /* Create a debug fs directory entry (root). */
> + sm_state->dir_root = debugfs_create_dir(VC_SM_DIR_ROOT_NAME,
> NULL);
> +
> + sm_state->dir_state.show = &vc_sm_cma_global_state_show;
> + sm_state->dir_state.dir_entry =
> + debugfs_create_file(VC_SM_STATE, 0444, sm_state-
> >dir_root,
> + &sm_state->dir_state,
> + &vc_sm_cma_debug_fs_fops);
> +
> + INIT_LIST_HEAD(&sm_state->buffer_list);
> +
> + /* Create a shared memory device. */
> + sm_state->misc_dev.minor = MISC_DYNAMIC_MINOR;
> + sm_state->misc_dev.name = DEVICE_NAME;
> + sm_state->misc_dev.fops = &vc_sm_ops;
> + sm_state->misc_dev.parent = NULL;
> + /* Temporarily set as 666 until udev rules have been sorted */
> + sm_state->misc_dev.mode = 0666;
> + ret = misc_register(&sm_state->misc_dev);
> + if (ret) {
> + pr_err("vcsm-cma: failed to register misc device.\n");
> + goto err_remove_debugfs;
> + }
> +
> + sm_state->data_knl = vc_sm_cma_create_priv_data(0);
> + if (!sm_state->data_knl) {
> + pr_err("[%s]: failed to create kernel private data
> tracker\n",
> + __func__);
> + goto err_remove_misc_dev;
> + }
> +
> + version.version = 2;
> + ret = vc_sm_cma_vchi_client_version(sm_state->sm_handle,
> &version,
> + &version_result,
> + &sm_state->int_trans_id);
> + if (ret) {
> + pr_err("[%s]: Failed to send version request %d\n",
> __func__,
> + ret);
> + }
> +
> + /* Done! */
> + sm_inited = 1;
> + pr_info("[%s]: installed successfully\n", __func__);
> + return;
> +
> +err_remove_misc_dev:
> + misc_deregister(&sm_state->misc_dev);
> +err_remove_debugfs:
> + debugfs_remove_recursive(sm_state->dir_root);
> + vc_sm_cma_vchi_stop(&sm_state->sm_handle);
> +}
> +
> +/* Driver loading. */
> +static int bcm2835_vc_sm_cma_probe(struct platform_device *pdev)
> +{
> + pr_info("%s: Videocore shared memory driver\n", __func__);
> +
> + sm_state = devm_kzalloc(&pdev->dev, sizeof(*sm_state),
> GFP_KERNEL);
> + if (!sm_state)
> + return -ENOMEM;
> + sm_state->pdev = pdev;
> + mutex_init(&sm_state->map_lock);
> +
> + spin_lock_init(&sm_state->kernelid_map_lock);
> + idr_init_base(&sm_state->kernelid_map, 1);
> +
> + pdev->dev.dma_parms = devm_kzalloc(&pdev->dev,
> + sizeof(*pdev-
> >dev.dma_parms),
> + GFP_KERNEL);
> + /* dma_set_max_seg_size checks if dma_parms is NULL. */
> + dma_set_max_seg_size(&pdev->dev, 0x3FFFFFFF);
> +
> + vchiq_add_connected_callback(vc_sm_connected_init);
> + return 0;
> +}
> +
> +/* Driver unloading. */
> +static int bcm2835_vc_sm_cma_remove(struct platform_device *pdev)
> +{
> + pr_debug("[%s]: start\n", __func__);
> + if (sm_inited) {
> + misc_deregister(&sm_state->misc_dev);
> +
> + /* Remove all proc entries. */
> + debugfs_remove_recursive(sm_state->dir_root);
> +
> + /* Stop the videocore shared memory service. */
> + vc_sm_cma_vchi_stop(&sm_state->sm_handle);
> + }
> +
> + if (sm_state) {
> + idr_destroy(&sm_state->kernelid_map);
> +
> + /* Free the memory for the state structure. */
> + mutex_destroy(&sm_state->map_lock);
> + }
> +
> + pr_debug("[%s]: end\n", __func__);
> + return 0;
> +}
> +
> +/* Kernel API calls */
> +/* Get an internal resource handle mapped from the external one. */
> +int vc_sm_cma_int_handle(void *handle)
> +{
> + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> + struct vc_sm_buffer *buf;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !handle) {
> + pr_err("[%s]: invalid input\n", __func__);
> + return 0;
> + }
> +
> + buf = (struct vc_sm_buffer *)dma_buf->priv;
> + return buf->vc_handle;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_int_handle);
> +
> +/* Free a previously allocated shared memory handle and block. */
> +int vc_sm_cma_free(void *handle)
> +{
> + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !handle) {
> + pr_err("[%s]: invalid input\n", __func__);
> + return -EPERM;
> + }
> +
> + pr_debug("%s: handle %p/dmabuf %p\n", __func__, handle,
> dma_buf);
> +
> + dma_buf_put(dma_buf);
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_free);
> +
> +/* Import a dmabuf to be shared with VC. */
> +int vc_sm_cma_import_dmabuf(struct dma_buf *src_dmabuf, void
> **handle)
> +{
> + struct dma_buf *new_dma_buf;
> + struct vc_sm_buffer *buf;
> + int ret;
> +
> + /* Validate we can work with this device. */
> + if (!sm_state || !src_dmabuf || !handle) {
> + pr_err("[%s]: invalid input\n", __func__);
> + return -EPERM;
> + }
> +
> + ret = vc_sm_cma_import_dmabuf_internal(sm_state->data_knl,
> src_dmabuf,
> + -1, &new_dma_buf);
> +
> + if (!ret) {
> + pr_debug("%s: imported to ptr %p\n", __func__,
> new_dma_buf);
> + buf = (struct vc_sm_buffer *)new_dma_buf->priv;
> +
> + /* Assign valid handle at this time.*/
> + *handle = new_dma_buf;
> + } else {
> + /*
> + * succeeded in importing the dma_buf, but then
> + * failed to look it up again. How?
> + * Release the fd again.
> + */
> + pr_err("%s: imported vc_sm_cma_get_buffer failed %d\n",
> + __func__, ret);
> + }
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(vc_sm_cma_import_dmabuf);
> +
> +static struct platform_driver bcm2835_vcsm_cma_driver = {
> + .probe = bcm2835_vc_sm_cma_probe,
> + .remove = bcm2835_vc_sm_cma_remove,
> + .driver = {
> + .name = DEVICE_NAME,
> + .owner = THIS_MODULE,
> + },
> +};
> +
> +module_platform_driver(bcm2835_vcsm_cma_driver);
> +
> +MODULE_AUTHOR("Dave Stevenson");
> +MODULE_DESCRIPTION("VideoCore CMA Shared Memory Driver");
> +MODULE_LICENSE("GPL v2");
> +MODULE_ALIAS("platform:vcsm-cma");
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> new file mode 100644
> index 000000000000..f1c7b95b14ce
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> @@ -0,0 +1,84 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory driver using CMA.
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + *
> + */
> +
> +#ifndef VC_SM_H
> +#define VC_SM_H
> +
> +#include <linux/device.h>
> +#include <linux/dma-direction.h>
> +#include <linux/kref.h>
> +#include <linux/mm_types.h>
> +#include <linux/mutex.h>
> +#include <linux/rbtree.h>
> +#include <linux/sched.h>
> +#include <linux/shrinker.h>
> +#include <linux/types.h>
> +#include <linux/miscdevice.h>
> +
> +#define VC_SM_MAX_NAME_LEN 32
> +
> +enum vc_sm_vpu_mapping_state {
> + VPU_NOT_MAPPED,
> + VPU_MAPPED,
> + VPU_UNMAPPING
> +};
> +
> +struct vc_sm_alloc_data {
> + unsigned long num_pages;
> + void *priv_virt;
> + struct sg_table *sg_table;
> +};
> +
> +struct vc_sm_imported {
> + struct dma_buf *dma_buf;
> + struct dma_buf_attachment *attach;
> + struct sg_table *sgt;
> +};
> +
> +struct vc_sm_buffer {
> + struct list_head global_buffer_list; /* Global list of
> buffers. */
> +
> + /* Index in the kernel_id idr so that we can find the
> + * mmal_msg_context again when servicing the VCHI reply.
> + */
> + int kernel_id;
> +
> + size_t size;
> +
> + /* Lock over all the following state for this buffer */
> + struct mutex lock;
> + struct list_head attachments;
> +
> + char name[VC_SM_MAX_NAME_LEN];
> +
> + int in_use:1; /* Kernel is still using this resource */
> + int imported:1; /* Imported dmabuf */
> +
> + enum vc_sm_vpu_mapping_state vpu_state;
> + u32 vc_handle; /* VideoCore handle for this buffer */
> + int vpu_allocated; /*
> + * The VPU made this allocation.
> Release the
> + * local dma_buf when the VPU releases
> the
> + * resource.
> + */
> +
> + /* DMABUF related fields */
> + struct dma_buf *dma_buf;
> + dma_addr_t dma_addr;
> + void *cookie;
> +
> + struct vc_sm_privdata_t *private;
> +
> + union {
> + struct vc_sm_alloc_data alloc;
> + struct vc_sm_imported import;
> + };
> +};
> +
> +#endif
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> new file mode 100644
> index 000000000000..6a203c60bf7f
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> @@ -0,0 +1,505 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> + *
> + * Based on vmcs_sm driver from Broadcom Corporation.
> + *
> + */
> +
> +/* ---- Include Files ----------------------------------------------
> ------- */
> +#include <linux/completion.h>
> +#include <linux/kernel.h>
> +#include <linux/kthread.h>
> +#include <linux/list.h>
> +#include <linux/mutex.h>
> +#include <linux/semaphore.h>
> +#include <linux/slab.h>
> +#include <linux/types.h>
> +
> +#include "vc_sm_cma_vchi.h"
> +
> +#define VC_SM_VER 1
> +#define VC_SM_MIN_VER 0
> +
> +/* ---- Private Constants and Types --------------------------------
> ------ */
> +
> +/* Command blocks come from a pool */
> +#define SM_MAX_NUM_CMD_RSP_BLKS 32
> +
> +struct sm_cmd_rsp_blk {
> + struct list_head head; /* To create lists */
> + /* To be signaled when the response is there */
> + struct completion cmplt;
> +
> + u32 id;
> + u16 length;
> +
> + u8 msg[VC_SM_MAX_MSG_LEN];
> +
> + uint32_t wait:1;
> + uint32_t sent:1;
> + uint32_t alloc:1;
> +
> +};
> +
> +struct sm_instance {
> + u32 num_connections;
> + struct vchi_service_handle
> *vchi_handle[VCHI_MAX_NUM_CONNECTIONS];
> + struct task_struct *io_thread;
> + struct completion io_cmplt;
> +
> + vpu_event_cb vpu_event;
> +
> + /* Mutex over the following lists */
> + struct mutex lock;
> + u32 trans_id;
> + struct list_head cmd_list;
> + struct list_head rsp_list;
> + struct list_head dead_list;
> +
> + struct sm_cmd_rsp_blk free_blk[SM_MAX_NUM_CMD_RSP_BLKS];
> +
> + /* Mutex over the free_list */
> + struct mutex free_lock;
> + struct list_head free_list;
> +
> + struct semaphore free_sema;
> +
> +};
> +
> +/* ---- Private Variables ------------------------------------------
> ------ */
> +
> +/* ---- Private Function Prototypes --------------------------------
> ------ */
> +
> +/* ---- Private Functions ------------------------------------------
> ------ */
> +static int
> +bcm2835_vchi_msg_queue(struct vchi_service_handle *handle,
> + void *data,
> + unsigned int size)
> +{
> + return vchi_queue_kernel_message(handle,
> + data,
> + size);
> +}
> +
> +static struct
> +sm_cmd_rsp_blk *vc_vchi_cmd_create(struct sm_instance *instance,
> + enum vc_sm_msg_type id, void *msg,
> + u32 size, int wait)
> +{
> + struct sm_cmd_rsp_blk *blk;
> + struct vc_sm_msg_hdr_t *hdr;
> +
> + if (down_interruptible(&instance->free_sema)) {
> + blk = kmalloc(sizeof(*blk), GFP_KERNEL);
> + if (!blk)
> + return NULL;
> +
> + blk->alloc = 1;
> + init_completion(&blk->cmplt);
> + } else {
> + mutex_lock(&instance->free_lock);
> + blk =
> + list_first_entry(&instance->free_list,
> + struct sm_cmd_rsp_blk, head);
> + list_del(&blk->head);
> + mutex_unlock(&instance->free_lock);
> + }
> +
> + blk->sent = 0;
> + blk->wait = wait;
> + blk->length = sizeof(*hdr) + size;
> +
> + hdr = (struct vc_sm_msg_hdr_t *)blk->msg;
> + hdr->type = id;
> + mutex_lock(&instance->lock);
> + instance->trans_id++;
> + /*
> + * Retain the top bit for identifying asynchronous events, or
> VPU cmds.
> + */
> + instance->trans_id &= ~0x80000000;
> + hdr->trans_id = instance->trans_id;
> + blk->id = instance->trans_id;
> + mutex_unlock(&instance->lock);
> +
> + if (size)
> + memcpy(hdr->body, msg, size);
> +
> + return blk;
> +}
> +
> +static void
> +vc_vchi_cmd_delete(struct sm_instance *instance, struct
> sm_cmd_rsp_blk *blk)
> +{
> + if (blk->alloc) {
> + kfree(blk);
> + return;
> + }
> +
> + mutex_lock(&instance->free_lock);
> + list_add(&blk->head, &instance->free_list);
> + mutex_unlock(&instance->free_lock);
> + up(&instance->free_sema);
> +}
> +
> +static void vc_sm_cma_vchi_rx_ack(struct sm_instance *instance,
> + struct sm_cmd_rsp_blk *cmd,
> + struct vc_sm_result_t *reply,
> + u32 reply_len)
> +{
> + mutex_lock(&instance->lock);
> + list_for_each_entry(cmd,
> + &instance->rsp_list,
> + head) {
> + if (cmd->id == reply->trans_id)
> + break;
> + }
> + mutex_unlock(&instance->lock);
> +
> + if (&cmd->head == &instance->rsp_list) {
> + //pr_debug("%s: received response %u, throw away...",
> + pr_err("%s: received response %u, throw away...",
> + __func__,
> + reply->trans_id);
> + } else if (reply_len > sizeof(cmd->msg)) {
> + pr_err("%s: reply too big (%u) %u, throw away...",
> + __func__, reply_len,
> + reply->trans_id);
> + } else {
> + memcpy(cmd->msg, reply,
> + reply_len);
> + complete(&cmd->cmplt);
> + }
> +}
> +
> +static int vc_sm_cma_vchi_videocore_io(void *arg)
> +{
> + struct sm_instance *instance = arg;
> + struct sm_cmd_rsp_blk *cmd = NULL, *cmd_tmp;
> + struct vc_sm_result_t *reply;
> + u32 reply_len;
> + s32 status;
> + int svc_use = 1;
> +
> + while (1) {
> + if (svc_use)
> + vchi_service_release(instance->vchi_handle[0]);
> + svc_use = 0;
> +
> + if (wait_for_completion_interruptible(&instance-
> >io_cmplt))
> + continue;
> +
> + vchi_service_use(instance->vchi_handle[0]);
> + svc_use = 1;
> +
> + do {
> + /*
> + * Get new command and move it to response list
> + */
> + mutex_lock(&instance->lock);
> + if (list_empty(&instance->cmd_list)) {
> + /* no more commands to process */
> + mutex_unlock(&instance->lock);
> + break;
> + }
> + cmd = list_first_entry(&instance->cmd_list,
> + struct sm_cmd_rsp_blk,
> head);
> + list_move(&cmd->head, &instance->rsp_list);
> + cmd->sent = 1;
> + mutex_unlock(&instance->lock);
> +
> + /* Send the command */
> + status =
> + bcm2835_vchi_msg_queue(instance-
> >vchi_handle[0],
> + cmd->msg, cmd-
> >length);
> + if (status) {
> + pr_err("%s: failed to queue message
> (%d)",
> + __func__, status);
> + }
> +
> + /* If no reply is needed then we're done */
> + if (!cmd->wait) {
> + mutex_lock(&instance->lock);
> + list_del(&cmd->head);
> + mutex_unlock(&instance->lock);
> + vc_vchi_cmd_delete(instance, cmd);
> + continue;
> + }
> +
> + if (status) {
> + complete(&cmd->cmplt);
> + continue;
> + }
> +
> + } while (1);
> +
> + while (!vchi_msg_peek(instance->vchi_handle[0], (void
> **)&reply,
> + &reply_len, VCHI_FLAGS_NONE)) {
> + if (reply->trans_id & 0x80000000) {
> + /* Async event or cmd from the VPU */
> + if (instance->vpu_event)
> + instance->vpu_event(instance,
> reply,
> + reply_len);
> + } else {
> + vc_sm_cma_vchi_rx_ack(instance, cmd,
> reply,
> + reply_len);
> + }
> +
> + vchi_msg_remove(instance->vchi_handle[0]);
> + }
> +
> + /* Go through the dead list and free them */
> + mutex_lock(&instance->lock);
> + list_for_each_entry_safe(cmd, cmd_tmp, &instance-
> >dead_list,
> + head) {
> + list_del(&cmd->head);
> + vc_vchi_cmd_delete(instance, cmd);
> + }
> + mutex_unlock(&instance->lock);
> + }
> +
> + return 0;
> +}
> +
> +static void vc_sm_cma_vchi_callback(void *param,
> + const enum vchi_callback_reason
> reason,
> + void *msg_handle)
> +{
> + struct sm_instance *instance = param;
> +
> + (void)msg_handle;
> +
> + switch (reason) {
> + case VCHI_CALLBACK_MSG_AVAILABLE:
> + complete(&instance->io_cmplt);
> + break;
> +
> + case VCHI_CALLBACK_SERVICE_CLOSED:
> + pr_info("%s: service CLOSED!!", __func__);
> + default:
> + break;
> + }
> +}
> +
> +struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle
> *vchi_instance,
> + unsigned int num_connections,
> + vpu_event_cb vpu_event)
> +{
> + u32 i;
> + struct sm_instance *instance;
> + int status;
> +
> + pr_debug("%s: start", __func__);
> +
> + if (num_connections > VCHI_MAX_NUM_CONNECTIONS) {
> + pr_err("%s: unsupported number of connections %u
> (max=%u)",
> + __func__, num_connections,
> VCHI_MAX_NUM_CONNECTIONS);
> +
> + goto err_null;
> + }
> + /* Allocate memory for this instance */
> + instance = kzalloc(sizeof(*instance), GFP_KERNEL);
> +
> + /* Misc initialisations */
> + mutex_init(&instance->lock);
> + init_completion(&instance->io_cmplt);
> + INIT_LIST_HEAD(&instance->cmd_list);
> + INIT_LIST_HEAD(&instance->rsp_list);
> + INIT_LIST_HEAD(&instance->dead_list);
> + INIT_LIST_HEAD(&instance->free_list);
> + sema_init(&instance->free_sema, SM_MAX_NUM_CMD_RSP_BLKS);
> + mutex_init(&instance->free_lock);
> + for (i = 0; i < SM_MAX_NUM_CMD_RSP_BLKS; i++) {
> + init_completion(&instance->free_blk[i].cmplt);
> + list_add(&instance->free_blk[i].head, &instance-
> >free_list);
> + }
> +
> + /* Open the VCHI service connections */
> + instance->num_connections = num_connections;
> + for (i = 0; i < num_connections; i++) {
> + struct service_creation params = {
> + .version = VCHI_VERSION_EX(VC_SM_VER,
> VC_SM_MIN_VER),
> + .service_id = VC_SM_SERVER_NAME,
> + .callback = vc_sm_cma_vchi_callback,
> + .callback_param = instance,
> + };
> +
> + status = vchi_service_open(vchi_instance,
> + ¶ms, &instance-
> >vchi_handle[i]);
> + if (status) {
> + pr_err("%s: failed to open VCHI service (%d)",
> + __func__, status);
> +
> + goto err_close_services;
> + }
> + }
> +
> + /* Create the thread which takes care of all io to/from
> videoocore. */
> + instance->io_thread =
> kthread_create(&vc_sm_cma_vchi_videocore_io,
> + (void *)instance, "SMIO");
> + if (!instance->io_thread) {
> + pr_err("%s: failed to create SMIO thread", __func__);
> +
> + goto err_close_services;
> + }
> + instance->vpu_event = vpu_event;
> + set_user_nice(instance->io_thread, -10);
> + wake_up_process(instance->io_thread);
> +
> + pr_debug("%s: success - instance %p", __func__, instance);
> + return instance;
> +
> +err_close_services:
> + for (i = 0; i < instance->num_connections; i++) {
> + if (instance->vchi_handle[i])
> + vchi_service_close(instance->vchi_handle[i]);
> + }
> + kfree(instance);
> +err_null:
> + pr_debug("%s: FAILED", __func__);
> + return NULL;
> +}
> +
> +int vc_sm_cma_vchi_stop(struct sm_instance **handle)
> +{
> + struct sm_instance *instance;
> + u32 i;
> +
> + if (!handle) {
> + pr_err("%s: invalid pointer to handle %p", __func__,
> handle);
> + goto lock;
> + }
> +
> + if (!*handle) {
> + pr_err("%s: invalid handle %p", __func__, *handle);
> + goto lock;
> + }
> +
> + instance = *handle;
> +
> + /* Close all VCHI service connections */
> + for (i = 0; i < instance->num_connections; i++) {
> + s32 success;
> +
> + vchi_service_use(instance->vchi_handle[i]);
> +
> + success = vchi_service_close(instance->vchi_handle[i]);
> + }
> +
> + kfree(instance);
> +
> + *handle = NULL;
> + return 0;
> +
> +lock:
> + return -EINVAL;
> +}
> +
> +static int vc_sm_cma_vchi_send_msg(struct sm_instance *handle,
> + enum vc_sm_msg_type msg_id, void
> *msg,
> + u32 msg_size, void *result, u32
> result_size,
> + u32 *cur_trans_id, u8 wait_reply)
> +{
> + int status = 0;
> + struct sm_instance *instance = handle;
> + struct sm_cmd_rsp_blk *cmd_blk;
> +
> + if (!handle) {
> + pr_err("%s: invalid handle", __func__);
> + return -EINVAL;
> + }
> + if (!msg) {
> + pr_err("%s: invalid msg pointer", __func__);
> + return -EINVAL;
> + }
> +
> + cmd_blk =
> + vc_vchi_cmd_create(instance, msg_id, msg, msg_size,
> wait_reply);
> + if (!cmd_blk) {
> + pr_err("[%s]: failed to allocate global tracking
> resource",
> + __func__);
> + return -ENOMEM;
> + }
> +
> + if (cur_trans_id)
> + *cur_trans_id = cmd_blk->id;
> +
> + mutex_lock(&instance->lock);
> + list_add_tail(&cmd_blk->head, &instance->cmd_list);
> + mutex_unlock(&instance->lock);
> + complete(&instance->io_cmplt);
> +
> + if (!wait_reply)
> + /* We're done */
> + return 0;
> +
> + /* Wait for the response */
> + if (wait_for_completion_interruptible(&cmd_blk->cmplt)) {
> + mutex_lock(&instance->lock);
> + if (!cmd_blk->sent) {
> + list_del(&cmd_blk->head);
> + mutex_unlock(&instance->lock);
> + vc_vchi_cmd_delete(instance, cmd_blk);
> + return -ENXIO;
> + }
> +
> + list_move(&cmd_blk->head, &instance->dead_list);
> + mutex_unlock(&instance->lock);
> + complete(&instance->io_cmplt);
> + return -EINTR; /* We're done */
> + }
> +
> + if (result && result_size) {
> + memcpy(result, cmd_blk->msg, result_size);
> + } else {
> + struct vc_sm_result_t *res =
> + (struct vc_sm_result_t *)cmd_blk->msg;
> + status = (res->success == 0) ? 0 : -ENXIO;
> + }
> +
> + mutex_lock(&instance->lock);
> + list_del(&cmd_blk->head);
> + mutex_unlock(&instance->lock);
> + vc_vchi_cmd_delete(instance, cmd_blk);
> + return status;
> +}
> +
> +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct
> vc_sm_free_t *msg,
> + u32 *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_FREE,
> + msg, sizeof(*msg), 0, 0,
> cur_trans_id, 0);
> +}
> +
> +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct
> vc_sm_import *msg,
> + struct vc_sm_import_result *result, u32
> *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_IMPORT,
> + msg, sizeof(*msg), result,
> sizeof(*result),
> + cur_trans_id, 1);
> +}
> +
> +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> + struct vc_sm_version *msg,
> + struct vc_sm_result_t *result,
> + u32 *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle,
> VC_SM_MSG_TYPE_CLIENT_VERSION,
> + //msg, sizeof(*msg), result,
> sizeof(*result),
> + //cur_trans_id, 1);
> + msg, sizeof(*msg), NULL, 0,
> + cur_trans_id, 0);
> +}
> +
> +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> + struct
> vc_sm_vc_mem_request_result *msg,
> + uint32_t *cur_trans_id)
> +{
> + return vc_sm_cma_vchi_send_msg(handle,
> + VC_SM_MSG_TYPE_VC_MEM_REQUEST_RE
> PLY,
> + msg, sizeof(*msg), 0, 0,
> cur_trans_id,
> + 0);
> +}
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> new file mode 100644
> index 000000000000..e8db34bd1e91
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> @@ -0,0 +1,63 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> + *
> + * Based on vmcs_sm driver from Broadcom Corporation.
> + *
> + */
> +
> +#ifndef __VC_SM_CMA_VCHI_H__INCLUDED__
> +#define __VC_SM_CMA_VCHI_H__INCLUDED__
> +
> +#include "interface/vchi/vchi.h"
> +
> +#include "vc_sm_defs.h"
> +
> +/*
> + * Forward declare.
> + */
> +struct sm_instance;
> +
> +typedef void (*vpu_event_cb)(struct sm_instance *instance,
> + struct vc_sm_result_t *reply, int
> reply_len);
> +
> +/*
> + * Initialize the shared memory service, opens up vchi connection to
> talk to it.
> + */
> +struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle
> *vchi_instance,
> + unsigned int num_connections,
> + vpu_event_cb vpu_event);
> +
> +/*
> + * Terminates the shared memory service.
> + */
> +int vc_sm_cma_vchi_stop(struct sm_instance **handle);
> +
> +/*
> + * Ask the shared memory service to free up some memory that was
> previously
> + * allocated by the vc_sm_cma_vchi_alloc function call.
> + */
> +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct
> vc_sm_free_t *msg,
> + u32 *cur_trans_id);
> +
> +/*
> + * Import a contiguous block of memory and wrap it in a GPU
> MEM_HANDLE_T.
> + */
> +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct
> vc_sm_import *msg,
> + struct vc_sm_import_result *result,
> + u32 *cur_trans_id);
> +
> +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> + struct vc_sm_version *msg,
> + struct vc_sm_result_t *result,
> + u32 *cur_trans_id);
> +
> +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> + struct
> vc_sm_vc_mem_request_result *msg,
> + uint32_t *cur_trans_id);
> +
> +#endif /* __VC_SM_CMA_VCHI_H__INCLUDED__ */
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> new file mode 100644
> index 000000000000..8a0d1f6dbfe8
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> @@ -0,0 +1,300 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + *
> + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom
> Corporation.
> + * All IPC messages are copied across to this file, even if the vc-
> sm-cma
> + * driver is not currently using them.
> + *
> +
> *********************************************************************
> *******
> + */
> +
> +#ifndef __VC_SM_DEFS_H__INCLUDED__
> +#define __VC_SM_DEFS_H__INCLUDED__
> +
> +/* FourCC code used for VCHI connection */
> +#define VC_SM_SERVER_NAME MAKE_FOURCC("SMEM")
> +
> +/* Maximum message length */
> +#define VC_SM_MAX_MSG_LEN (sizeof(union vc_sm_msg_union_t) + \
> + sizeof(struct vc_sm_msg_hdr_t))
> +#define VC_SM_MAX_RSP_LEN (sizeof(union vc_sm_msg_union_t))
> +
> +/* Resource name maximum size */
> +#define VC_SM_RESOURCE_NAME 32
> +
> +/*
> + * Version to be reported to the VPU
> + * VPU assumes 0 (aka 1) which does not require the released
> callback, nor
> + * expect the client to handle VC_MEM_REQUESTS.
> + * Version 2 requires the released callback, and must support
> VC_MEM_REQUESTS.
> + */
> +#define VC_SM_PROTOCOL_VERSION 2
> +
> +enum vc_sm_msg_type {
> + /* Message types supported for HOST->VC direction */
> +
> + /* Allocate shared memory block */
> + VC_SM_MSG_TYPE_ALLOC,
> + /* Lock allocated shared memory block */
> + VC_SM_MSG_TYPE_LOCK,
> + /* Unlock allocated shared memory block */
> + VC_SM_MSG_TYPE_UNLOCK,
> + /* Unlock allocated shared memory block, do not answer command
> */
> + VC_SM_MSG_TYPE_UNLOCK_NOANS,
> + /* Free shared memory block */
> + VC_SM_MSG_TYPE_FREE,
> + /* Resize a shared memory block */
> + VC_SM_MSG_TYPE_RESIZE,
> + /* Walk the allocated shared memory block(s) */
> + VC_SM_MSG_TYPE_WALK_ALLOC,
> +
> + /* A previously applied action will need to be reverted */
> + VC_SM_MSG_TYPE_ACTION_CLEAN,
> +
> + /*
> + * Import a physical address and wrap into a MEM_HANDLE_T.
> + * Release with VC_SM_MSG_TYPE_FREE.
> + */
> + VC_SM_MSG_TYPE_IMPORT,
> + /*
> + *Tells VC the protocol version supported by this client.
> + * 2 supports the async/cmd messages from the VPU for final
> release
> + * of memory, and for VC allocations.
> + */
> + VC_SM_MSG_TYPE_CLIENT_VERSION,
> + /* Response to VC request for memory */
> + VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
> +
> + /*
> + * Asynchronous/cmd messages supported for VC->HOST direction.
> + * Signalled by setting the top bit in vc_sm_result_t trans_id.
> + */
> +
> + /*
> + * VC has finished with an imported memory allocation.
> + * Release any Linux reference counts on the underlying block.
> + */
> + VC_SM_MSG_TYPE_RELEASED,
> + /* VC request for memory */
> + VC_SM_MSG_TYPE_VC_MEM_REQUEST,
> +
> + VC_SM_MSG_TYPE_MAX
> +};
> +
> +/* Type of memory to be allocated */
> +enum vc_sm_alloc_type_t {
> + VC_SM_ALLOC_CACHED,
> + VC_SM_ALLOC_NON_CACHED,
> +};
> +
> +/* Message header for all messages in HOST->VC direction */
> +struct vc_sm_msg_hdr_t {
> + u32 type;
> + u32 trans_id;
> + u8 body[0];
> +
> +};
> +
> +/* Request to allocate memory (HOST->VC) */
> +struct vc_sm_alloc_t {
> + /* type of memory to allocate */
> + enum vc_sm_alloc_type_t type;
> + /* byte amount of data to allocate per unit */
> + u32 base_unit;
> + /* number of unit to allocate */
> + u32 num_unit;
> + /* alignment to be applied on allocation */
> + u32 alignment;
> + /* identity of who allocated this block */
> + u32 allocator;
> + /* resource name (for easier tracking on vc side) */
> + char name[VC_SM_RESOURCE_NAME];
> +
> +};
> +
> +/* Result of a requested memory allocation (VC->HOST) */
> +struct vc_sm_alloc_result_t {
> + /* Transaction identifier */
> + u32 trans_id;
> +
> + /* Resource handle */
> + u32 res_handle;
> + /* Pointer to resource buffer */
> + u32 res_mem;
> + /* Resource base size (bytes) */
> + u32 res_base_size;
> + /* Resource number */
> + u32 res_num;
> +
> +};
> +
> +/* Request to free a previously allocated memory (HOST->VC) */
> +struct vc_sm_free_t {
> + /* Resource handle (returned from alloc) */
> + u32 res_handle;
> + /* Resource buffer (returned from alloc) */
> + u32 res_mem;
> +
> +};
> +
> +/* Request to lock a previously allocated memory (HOST->VC) */
> +struct vc_sm_lock_unlock_t {
> + /* Resource handle (returned from alloc) */
> + u32 res_handle;
> + /* Resource buffer (returned from alloc) */
> + u32 res_mem;
> +
> +};
> +
> +/* Request to resize a previously allocated memory (HOST->VC) */
> +struct vc_sm_resize_t {
> + /* Resource handle (returned from alloc) */
> + u32 res_handle;
> + /* Resource buffer (returned from alloc) */
> + u32 res_mem;
> + /* Resource *new* size requested (bytes) */
> + u32 res_new_size;
> +
> +};
> +
> +/* Result of a requested memory lock (VC->HOST) */
> +struct vc_sm_lock_result_t {
> + /* Transaction identifier */
> + u32 trans_id;
> +
> + /* Resource handle */
> + u32 res_handle;
> + /* Pointer to resource buffer */
> + u32 res_mem;
> + /*
> + * Pointer to former resource buffer if the memory
> + * was reallocated
> + */
> + u32 res_old_mem;
> +
> +};
> +
> +/* Generic result for a request (VC->HOST) */
> +struct vc_sm_result_t {
> + /* Transaction identifier */
> + u32 trans_id;
> +
> + s32 success;
> +
> +};
> +
> +/* Request to revert a previously applied action (HOST->VC) */
> +struct vc_sm_action_clean_t {
> + /* Action of interest */
> + enum vc_sm_msg_type res_action;
> + /* Transaction identifier for the action of interest */
> + u32 action_trans_id;
> +
> +};
> +
> +/* Request to remove all data associated with a given allocator
> (HOST->VC) */
> +struct vc_sm_free_all_t {
> + /* Allocator identifier */
> + u32 allocator;
> +};
> +
> +/* Request to import memory (HOST->VC) */
> +struct vc_sm_import {
> + /* type of memory to allocate */
> + enum vc_sm_alloc_type_t type;
> + /* pointer to the VC (ie physical) address of the allocated
> memory */
> + u32 addr;
> + /* size of buffer */
> + u32 size;
> + /* opaque handle returned in RELEASED messages */
> + u32 kernel_id;
> + /* Allocator identifier */
> + u32 allocator;
> + /* resource name (for easier tracking on vc side) */
> + char name[VC_SM_RESOURCE_NAME];
> +};
> +
> +/* Result of a requested memory import (VC->HOST) */
> +struct vc_sm_import_result {
> + /* Transaction identifier */
> + u32 trans_id;
> +
> + /* Resource handle */
> + u32 res_handle;
> +};
> +
> +/* Notification that VC has finished with an allocation (VC->HOST)
> */
> +struct vc_sm_released {
> + /* cmd type / trans_id */
> + u32 cmd;
> +
> + /* pointer to the VC (ie physical) address of the allocated
> memory */
> + u32 addr;
> + /* size of buffer */
> + u32 size;
> + /* opaque handle returned in RELEASED messages */
> + u32 kernel_id;
> + u32 vc_handle;
> +};
> +
> +/*
> + * Client informing VC as to the protocol version it supports.
> + * >=2 requires the released callback, and supports VC asking for
> memory.
> + * Failure means that the firmware doesn't support this call, and
> therefore the
> + * client should either fail, or NOT rely on getting the released
> callback.
> + */
> +struct vc_sm_version {
> + u32 version;
> +};
> +
> +/* Request FROM VideoCore for some memory */
> +struct vc_sm_vc_mem_request {
> + /* cmd type */
> + u32 cmd;
> +
> + /* trans_id (from VPU) */
> + u32 trans_id;
> + /* size of buffer */
> + u32 size;
> + /* alignment of buffer */
> + u32 align;
> + /* resource name (for easier tracking) */
> + char name[VC_SM_RESOURCE_NAME];
> + /* VPU handle for the resource */
> + u32 vc_handle;
> +};
> +
> +/* Response from the kernel to provide the VPU with some memory */
> +struct vc_sm_vc_mem_request_result {
> + /* Transaction identifier for the VPU */
> + u32 trans_id;
> + /* pointer to the physical address of the allocated memory */
> + u32 addr;
> + /* opaque handle returned in RELEASED messages */
> + u32 kernel_id;
> +};
> +
> +/* Union of ALL messages */
> +union vc_sm_msg_union_t {
> + struct vc_sm_alloc_t alloc;
> + struct vc_sm_alloc_result_t alloc_result;
> + struct vc_sm_free_t free;
> + struct vc_sm_lock_unlock_t lock_unlock;
> + struct vc_sm_action_clean_t action_clean;
> + struct vc_sm_resize_t resize;
> + struct vc_sm_lock_result_t lock_result;
> + struct vc_sm_result_t result;
> + struct vc_sm_free_all_t free_all;
> + struct vc_sm_import import;
> + struct vc_sm_import_result import_result;
> + struct vc_sm_version version;
> + struct vc_sm_released released;
> + struct vc_sm_vc_mem_request vc_request;
> + struct vc_sm_vc_mem_request_result vc_request_result;
> +};
> +
> +#endif /* __VC_SM_DEFS_H__INCLUDED__ */
> diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> new file mode 100644
> index 000000000000..988fdd967922
> --- /dev/null
> +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> @@ -0,0 +1,28 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +
> +/*
> + * VideoCore Shared Memory CMA allocator
> + *
> + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> + *
> + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom
> Corporation.
> + *
> + */
> +
> +#ifndef __VC_SM_KNL_H__INCLUDED__
> +#define __VC_SM_KNL_H__INCLUDED__
> +
> +#if !defined(__KERNEL__)
> +#error "This interface is for kernel use only..."
> +#endif
> +
> +/* Free a previously allocated or imported shared memory handle and
> block. */
> +int vc_sm_cma_free(void *handle);
> +
> +/* Get an internal resource handle mapped from the external one. */
> +int vc_sm_cma_int_handle(void *handle);
> +
> +/* Import a block of memory into the GPU space. */
> +int vc_sm_cma_import_dmabuf(struct dma_buf *dmabuf, void **handle);
> +
> +#endif /* __VC_SM_KNL_H__INCLUDED__ */
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-06 18:04 ` Nicolas Saenz Julienne
@ 2020-05-06 19:24 ` Dave Stevenson
2020-05-08 0:11 ` Laurent Pinchart
2020-08-24 16:39 ` Jacopo Mondi
0 siblings, 2 replies; 111+ messages in thread
From: Dave Stevenson @ 2020-05-06 19:24 UTC (permalink / raw)
To: Nicolas Saenz Julienne
Cc: Laurent Pinchart, Linux Media Mailing List, Dave Stevenson,
Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck
Hi Nicolas
On Wed, 6 May 2020 at 19:04, Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Hi Laurent, Dave,
>
> On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> >
> > Add Broadcom VideoCore Shared Memory support.
> >
> > This new driver allows contiguous memory blocks to be imported
> > into the VideoCore VPU memory map, and manages the lifetime of
> > those objects, only releasing the source dmabuf once the VPU has
> > confirmed it has finished with it.
> >
>
> I'm still digesting all this, but a question came up, who is using the
> ioctls?
We have a userspace library that uses it [1].
It is used by things like MMAL to share buffers between the VPU and
ARM, rather than having to get VCHI to copy all the data between
mirrored buffers.
I think what has happened here is that Laurent has picked up the
version of the driver from the top of our downstream kernel tree.
For libcamera and the ISP driver, we need a significantly smaller
feature set, basically import of dmabufs only, no allocations or cache
management. For the ISP driver it's mainly dmabuf import from
videobuf2 for the image buffers, but there's also a need to pass in
lens shading tables which are relatively large. With a small amount of
rework in libcamera, we can make it so that we use dma-buf heaps to do
the allocation, and pass in a dmabuf fd to the ISP driver to then map
onto the VPU. That removes all the ioctls handling from this driver.
Downstream we do have other use cases that want to be able to do other
functions on shared memory, but that too should be reworkable into
using dma-buf heaps for allocations, and vcsm only handles importing
dmabufs via an ioctl. All that can be hidden away in the vcsm library,
so applications don't care.
We've also got some legacy code kicking around, as there was
originally a version of the driver that mapped the VPU's memory blocks
to the ARM. That's why the vcsm library has two code paths through
almost every function - one for each driver.
Laurent: What's your view? Halt the review this particular patch for
now and rework, or try and get this all integrated?
Mainline obviously already has dma-buf heaps merged, whilst I have a
PR cherry-picking it back into our downstream 5.4. The main reason it
hasn't been merged is that I haven't had a test case to prove it
works. The rework should be relatively simple, but will need small
updates to both libcamera and ISP driver.
Dave
[1] https://github.com/raspberrypi/userland/tree/master/host_applications/linux/libs/sm
> Regards,
> Nicolas
>
> > Driver upported from the RaspberryPi BSP at revision:
> > 890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when
> > counting transactions")
> > forward ported to recent mainline kernel version.
> >
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> > drivers/staging/vc04_services/Kconfig | 2 +
> > drivers/staging/vc04_services/Makefile | 1 +
> > .../include/linux/broadcom/vc_sm_cma_ioctl.h | 114 ++
> > .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> > .../staging/vc04_services/vc-sm-cma/Makefile | 13 +
> > drivers/staging/vc04_services/vc-sm-cma/TODO | 1 +
> > .../staging/vc04_services/vc-sm-cma/vc_sm.c | 1732
> > +++++++++++++++++
> > .../staging/vc04_services/vc-sm-cma/vc_sm.h | 84 +
> > .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 505 +++++
> > .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
> > .../vc04_services/vc-sm-cma/vc_sm_defs.h | 300 +++
> > .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> > 12 files changed, 2853 insertions(+)
> > create mode 100644
> > drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.
> > h
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/TODO
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > cma/vc_sm_cma_vchi.c
> > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > cma/vc_sm_cma_vchi.h
> > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > cma/vc_sm_defs.h
> > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > cma/vc_sm_knl.h
> >
> > diff --git a/drivers/staging/vc04_services/Kconfig
> > b/drivers/staging/vc04_services/Kconfig
> > index 6baf9dd57f1f..b9f1f019ebd8 100644
> > --- a/drivers/staging/vc04_services/Kconfig
> > +++ b/drivers/staging/vc04_services/Kconfig
> > @@ -23,5 +23,7 @@ source "drivers/staging/vc04_services/bcm2835-
> > audio/Kconfig"
> >
> > source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
> >
> > +source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
> > +
> > endif
> >
> > diff --git a/drivers/staging/vc04_services/Makefile
> > b/drivers/staging/vc04_services/Makefile
> > index 54d9e2f31916..6e1abf494c1a 100644
> > --- a/drivers/staging/vc04_services/Makefile
> > +++ b/drivers/staging/vc04_services/Makefile
> > @@ -12,6 +12,7 @@ vchiq-objs := \
> >
> > obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
> > obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
> > +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
> >
> > ccflags-y += -D__VCCOREVER__=0x04000000
> >
> > diff --git
> > a/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> > l.h
> > b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> > l.h
> > new file mode 100644
> > index 000000000000..107460ad1be3
> > --- /dev/null
> > +++
> > b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> > l.h
> > @@ -0,0 +1,114 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +/*
> > + * Copyright 2019 Raspberry Pi (Trading) Ltd. All rights reserved.
> > + *
> > + * Based on vmcs_sm_ioctl.h Copyright Broadcom Corporation.
> > + */
> > +
> > +#ifndef __VC_SM_CMA_IOCTL_H
> > +#define __VC_SM_CMA_IOCTL_H
> > +
> > +/* ---- Include Files ----------------------------------------------
> > ------ */
> > +
> > +#if defined(__KERNEL__)
> > +#include <linux/types.h> /* Needed for standard types */
> > +#else
> > +#include <stdint.h>
> > +#endif
> > +
> > +#include <linux/ioctl.h>
> > +
> > +/* ---- Constants and Types ----------------------------------------
> > ------ */
> > +
> > +#define VC_SM_CMA_RESOURCE_NAME 32
> > +#define VC_SM_CMA_RESOURCE_NAME_DEFAULT "sm-host-resource"
> > +
> > +/* Type define used to create unique IOCTL number */
> > +#define VC_SM_CMA_MAGIC_TYPE 'J'
> > +
> > +/* IOCTL commands on /dev/vc-sm-cma */
> > +enum vc_sm_cma_cmd_e {
> > + VC_SM_CMA_CMD_ALLOC = 0x5A, /* Start at 0x5A arbitrarily
> > */
> > +
> > + VC_SM_CMA_CMD_IMPORT_DMABUF,
> > +
> > + VC_SM_CMA_CMD_CLEAN_INVALID2,
> > +
> > + VC_SM_CMA_CMD_LAST /* Do not delete */
> > +};
> > +
> > +/* Cache type supported, conveniently matches the user space
> > definition in
> > + * user-vcsm.h.
> > + */
> > +enum vc_sm_cma_cache_e {
> > + VC_SM_CMA_CACHE_NONE,
> > + VC_SM_CMA_CACHE_HOST,
> > + VC_SM_CMA_CACHE_VC,
> > + VC_SM_CMA_CACHE_BOTH,
> > +};
> > +
> > +/* IOCTL Data structures */
> > +struct vc_sm_cma_ioctl_alloc {
> > + /* user -> kernel */
> > + __u32 size;
> > + __u32 num;
> > + __u32 cached; /* enum vc_sm_cma_cache_e */
> > + __u32 pad;
> > + __u8 name[VC_SM_CMA_RESOURCE_NAME];
> > +
> > + /* kernel -> user */
> > + __s32 handle;
> > + __u32 vc_handle;
> > + __u64 dma_addr;
> > +};
> > +
> > +struct vc_sm_cma_ioctl_import_dmabuf {
> > + /* user -> kernel */
> > + __s32 dmabuf_fd;
> > + __u32 cached; /* enum vc_sm_cma_cache_e */
> > + __u8 name[VC_SM_CMA_RESOURCE_NAME];
> > +
> > + /* kernel -> user */
> > + __s32 handle;
> > + __u32 vc_handle;
> > + __u32 size;
> > + __u32 pad;
> > + __u64 dma_addr;
> > +};
> > +
> > +/*
> > + * Cache functions to be set to struct
> > vc_sm_cma_ioctl_clean_invalid2
> > + * invalidate_mode.
> > + */
> > +#define VC_SM_CACHE_OP_NOP 0x00
> > +#define VC_SM_CACHE_OP_INV 0x01
> > +#define VC_SM_CACHE_OP_CLEAN 0x02
> > +#define VC_SM_CACHE_OP_FLUSH 0x03
> > +
> > +struct vc_sm_cma_ioctl_clean_invalid2 {
> > + __u32 op_count;
> > + __u32 pad;
> > + struct vc_sm_cma_ioctl_clean_invalid_block {
> > + __u32 invalidate_mode;
> > + __u32 block_count;
> > + void * __user start_address;
> > + __u32 block_size;
> > + __u32 inter_block_stride;
> > + } s[0];
> > +};
> > +
> > +/* IOCTL numbers */
> > +#define VC_SM_CMA_IOCTL_MEM_ALLOC\
> > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_ALLOC,\
> > + struct vc_sm_cma_ioctl_alloc)
> > +
> > +#define VC_SM_CMA_IOCTL_MEM_IMPORT_DMABUF\
> > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_IMPORT_DMABUF,\
> > + struct vc_sm_cma_ioctl_import_dmabuf)
> > +
> > +#define VC_SM_CMA_IOCTL_MEM_CLEAN_INVALID2\
> > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
> > + struct vc_sm_cma_ioctl_clean_invalid2)
> > +
> > +#endif /* __VC_SM_CMA_IOCTL_H */
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > new file mode 100644
> > index 000000000000..5ac115da6b49
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > @@ -0,0 +1,10 @@
> > +config BCM_VC_SM_CMA
> > + bool "VideoCore Shared Memory (CMA) driver"
> > + select BCM2835_VCHIQ
> > + select RBTREE
> > + select DMA_SHARED_BUFFER
> > + help
> > + Say Y here to enable the shared memory interface that
> > + supports sharing dmabufs with VideoCore.
> > + This operates over the VCHIQ interface to a service
> > + running on VideoCore.
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/Makefile
> > b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> > new file mode 100644
> > index 000000000000..77d173694fbf
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> > @@ -0,0 +1,13 @@
> > +ccflags-y += \
> > + -I$(srctree)/$(src)/../ \
> > + -I$(srctree)/$(src)/../interface/vchi \
> > + -I$(srctree)/$(src)/../interface/vchiq_arm\
> > + -I$(srctree)/$(src)/../include
> > +
> > +ccflags-y += \
> > + -D__VCCOREVER__=0
> > +
> > +vc-sm-cma-$(CONFIG_BCM_VC_SM_CMA) := \
> > + vc_sm.o vc_sm_cma_vchi.o
> > +
> > +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma.o
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/TODO
> > b/drivers/staging/vc04_services/vc-sm-cma/TODO
> > new file mode 100644
> > index 000000000000..ac9b5f8a7389
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/TODO
> > @@ -0,0 +1 @@
> > +No currently outstanding tasks except some clean-up.
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > new file mode 100644
> > index 000000000000..cd5fb561debb
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > @@ -0,0 +1,1732 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * VideoCore Shared Memory driver using CMA.
> > + *
> > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > + * Dave Stevenson <dave.stevenson@raspberrypi.org>
> > + *
> > + * Based on vmcs_sm driver from Broadcom Corporation for some API,
> > + * and taking some code for buffer allocation and dmabuf handling
> > from
> > + * videobuf2.
> > + *
> > + *
> > + * This driver has 3 main uses:
> > + * 1) Allocating buffers for the kernel or userspace that can be
> > shared with the
> > + * VPU.
> > + * 2) Importing dmabufs from elsewhere for sharing with the VPU.
> > + * 3) Allocating buffers for use by the VPU.
> > + *
> > + * In the first and second cases the native handle is a dmabuf.
> > Releasing the
> > + * resource inherently comes from releasing the dmabuf, and this
> > will trigger
> > + * unmapping on the VPU. The underlying allocation and our buffer
> > structure are
> > + * retained until the VPU has confirmed that it has finished with
> > it.
> > + *
> > + * For the VPU allocations the VPU is responsible for triggering the
> > release,
> > + * and therefore the released message decrements the dma_buf
> > refcount (with the
> > + * VPU mapping having already been marked as released).
> > + */
> > +
> > +/* ---- Include Files ----------------------------------------------
> > ------- */
> > +#include <linux/cdev.h>
> > +#include <linux/device.h>
> > +#include <linux/debugfs.h>
> > +#include <linux/dma-mapping.h>
> > +#include <linux/dma-buf.h>
> > +#include <linux/errno.h>
> > +#include <linux/fs.h>
> > +#include <linux/kernel.h>
> > +#include <linux/list.h>
> > +#include <linux/miscdevice.h>
> > +#include <linux/module.h>
> > +#include <linux/mm.h>
> > +#include <linux/of_device.h>
> > +#include <linux/platform_device.h>
> > +#include <linux/proc_fs.h>
> > +#include <linux/slab.h>
> > +#include <linux/seq_file.h>
> > +#include <linux/syscalls.h>
> > +#include <linux/types.h>
> > +#include <asm/cacheflush.h>
> > +
> > +#include "vchiq_connected.h"
> > +#include "vc_sm_cma_vchi.h"
> > +
> > +#include "vc_sm.h"
> > +#include "vc_sm_knl.h"
> > +#include <linux/broadcom/vc_sm_cma_ioctl.h>
> > +
> > +/* ---- Private Constants and Types --------------------------------
> > ------- */
> > +
> > +#define DEVICE_NAME "vcsm-cma"
> > +#define DEVICE_MINOR 0
> > +
> > +#define VC_SM_RESOURCE_NAME_DEFAULT "sm-host-resource"
> > +
> > +#define VC_SM_DIR_ROOT_NAME "vcsm-cma"
> > +#define VC_SM_STATE "state"
> > +
> > +/* Private file data associated with each opened device. */
> > +struct vc_sm_privdata_t {
> > + pid_t pid; /* PID of creator. */
> > +
> > + int restart_sys; /* Tracks restart on interrupt. */
> > + enum vc_sm_msg_type int_action; /* Interrupted action. */
> > + u32 int_trans_id; /* Interrupted transaction. */
> > +};
> > +
> > +typedef int (*VC_SM_SHOW) (struct seq_file *s, void *v);
> > +struct sm_pde_t {
> > + VC_SM_SHOW show; /* Debug fs function hookup. */
> > + struct dentry *dir_entry; /* Debug fs directory entry. */
> > + void *priv_data; /* Private data */
> > +};
> > +
> > +/* Global state information. */
> > +struct sm_state_t {
> > + struct platform_device *pdev;
> > +
> > + struct miscdevice misc_dev;
> > +
> > + struct sm_instance *sm_handle; /* Handle for videocore
> > service. */
> > +
> > + spinlock_t kernelid_map_lock; /* Spinlock protecting
> > kernelid_map */
> > + struct idr kernelid_map;
> > +
> > + struct mutex map_lock; /* Global map lock. */
> > + struct list_head buffer_list; /* List of buffer. */
> > +
> > + struct vc_sm_privdata_t *data_knl; /* Kernel internal data
> > tracking. */
> > + struct vc_sm_privdata_t *vpu_allocs; /* All allocations from
> > the VPU */
> > + struct dentry *dir_root; /* Debug fs entries root. */
> > + struct sm_pde_t dir_state; /* Debug fs entries state
> > sub-tree. */
> > +
> > + bool require_released_callback; /* VPU will send a released
> > msg when it
> > + * has finished with a
> > resource.
> > + */
> > + u32 int_trans_id; /* Interrupted transaction. */
> > +};
> > +
> > +struct vc_sm_dma_buf_attachment {
> > + struct device *dev;
> > + struct sg_table sg_table;
> > + struct list_head list;
> > + enum dma_data_direction dma_dir;
> > +};
> > +
> > +/* ---- Private Variables ------------------------------------------
> > ----- */
> > +
> > +static struct sm_state_t *sm_state;
> > +static int sm_inited;
> > +
> > +/* ---- Private Function Prototypes --------------------------------
> > ------ */
> > +
> > +/* ---- Private Functions ------------------------------------------
> > ------ */
> > +
> > +static int get_kernel_id(struct vc_sm_buffer *buffer)
> > +{
> > + int handle;
> > +
> > + spin_lock(&sm_state->kernelid_map_lock);
> > + handle = idr_alloc(&sm_state->kernelid_map, buffer, 0, 0,
> > GFP_KERNEL);
> > + spin_unlock(&sm_state->kernelid_map_lock);
> > +
> > + return handle;
> > +}
> > +
> > +static struct vc_sm_buffer *lookup_kernel_id(int handle)
> > +{
> > + return idr_find(&sm_state->kernelid_map, handle);
> > +}
> > +
> > +static void free_kernel_id(int handle)
> > +{
> > + spin_lock(&sm_state->kernelid_map_lock);
> > + idr_remove(&sm_state->kernelid_map, handle);
> > + spin_unlock(&sm_state->kernelid_map_lock);
> > +}
> > +
> > +static int vc_sm_cma_seq_file_show(struct seq_file *s, void *v)
> > +{
> > + struct sm_pde_t *sm_pde;
> > +
> > + sm_pde = (struct sm_pde_t *)(s->private);
> > +
> > + if (sm_pde && sm_pde->show)
> > + sm_pde->show(s, v);
> > +
> > + return 0;
> > +}
> > +
> > +static int vc_sm_cma_single_open(struct inode *inode, struct file
> > *file)
> > +{
> > + return single_open(file, vc_sm_cma_seq_file_show, inode-
> > >i_private);
> > +}
> > +
> > +static const struct file_operations vc_sm_cma_debug_fs_fops = {
> > + .open = vc_sm_cma_single_open,
> > + .read = seq_read,
> > + .llseek = seq_lseek,
> > + .release = single_release,
> > +};
> > +
> > +static int vc_sm_cma_global_state_show(struct seq_file *s, void *v)
> > +{
> > + struct vc_sm_buffer *resource = NULL;
> > + int resource_count = 0;
> > +
> > + if (!sm_state)
> > + return 0;
> > +
> > + seq_printf(s, "\nVC-ServiceHandle %p\n", sm_state-
> > >sm_handle);
> > +
> > + /* Log all applicable mapping(s). */
> > +
> > + mutex_lock(&sm_state->map_lock);
> > + seq_puts(s, "\nResources\n");
> > + if (!list_empty(&sm_state->buffer_list)) {
> > + list_for_each_entry(resource, &sm_state->buffer_list,
> > + global_buffer_list) {
> > + resource_count++;
> > +
> > + seq_printf(s, "\nResource %p\n",
> > + resource);
> > + seq_printf(s, " NAME %s\n",
> > + resource->name);
> > + seq_printf(s, " SIZE %zu\n",
> > + resource->size);
> > + seq_printf(s, " DMABUF %p\n",
> > + resource->dma_buf);
> > + if (resource->imported) {
> > + seq_printf(s,
> > " ATTACH %p\n",
> > + resource->import.attach);
> > + seq_printf(s,
> > " SGT %p\n",
> > + resource->import.sgt);
> > + } else {
> > + seq_printf(s,
> > " SGT %p\n",
> > + resource->alloc.sg_table);
> > + }
> > + seq_printf(s, " DMA_ADDR %pad\n",
> > + &resource->dma_addr);
> > + seq_printf(s,
> > " VC_HANDLE %08x\n",
> > + resource->vc_handle);
> > + seq_printf(s, " VC_MAPPING %d\n",
> > + resource->vpu_state);
> > + }
> > + }
> > + seq_printf(s, "\n\nTotal resource count: %d\n\n",
> > resource_count);
> > +
> > + mutex_unlock(&sm_state->map_lock);
> > +
> > + return 0;
> > +}
> > +
> > +/*
> > + * Adds a buffer to the private data list which tracks all the
> > allocated
> > + * data.
> > + */
> > +static void vc_sm_add_resource(struct vc_sm_privdata_t *privdata,
> > + struct vc_sm_buffer *buffer)
> > +{
> > + mutex_lock(&sm_state->map_lock);
> > + list_add(&buffer->global_buffer_list, &sm_state->buffer_list);
> > + mutex_unlock(&sm_state->map_lock);
> > +
> > + pr_debug("[%s]: added buffer %p (name %s, size %zu)\n",
> > + __func__, buffer, buffer->name, buffer->size);
> > +}
> > +
> > +/*
> > + * Cleans up imported dmabuf.
> > + */
> > +static void vc_sm_clean_up_dmabuf(struct vc_sm_buffer *buffer)
> > +{
> > + if (!buffer->imported)
> > + return;
> > +
> > + /* Handle cleaning up imported dmabufs */
> > + mutex_lock(&buffer->lock);
> > + if (buffer->import.sgt) {
> > + dma_buf_unmap_attachment(buffer->import.attach,
> > + buffer->import.sgt,
> > + DMA_BIDIRECTIONAL);
> > + buffer->import.sgt = NULL;
> > + }
> > + if (buffer->import.attach) {
> > + dma_buf_detach(buffer->dma_buf, buffer->import.attach);
> > + buffer->import.attach = NULL;
> > + }
> > + mutex_unlock(&buffer->lock);
> > +}
> > +
> > +/*
> > + * Instructs VPU to decrement the refcount on a buffer.
> > + */
> > +static void vc_sm_vpu_free(struct vc_sm_buffer *buffer)
> > +{
> > + if (buffer->vc_handle && buffer->vpu_state == VPU_MAPPED) {
> > + struct vc_sm_free_t free = { buffer->vc_handle, 0 };
> > + int status = vc_sm_cma_vchi_free(sm_state->sm_handle,
> > &free,
> > + &sm_state->int_trans_id);
> > + if (status != 0 && status != -EINTR) {
> > + pr_err("[%s]: failed to free memory on
> > videocore (status: %u, trans_id: %u)\n",
> > + __func__, status, sm_state-
> > >int_trans_id);
> > + }
> > +
> > + if (sm_state->require_released_callback) {
> > + /* Need to wait for the VPU to confirm the
> > free. */
> > +
> > + /* Retain a reference on this until the VPU has
> > + * released it
> > + */
> > + buffer->vpu_state = VPU_UNMAPPING;
> > + } else {
> > + buffer->vpu_state = VPU_NOT_MAPPED;
> > + buffer->vc_handle = 0;
> > + }
> > + }
> > +}
> > +
> > +/*
> > + * Release an allocation.
> > + * All refcounting is done via the dma buf object.
> > + *
> > + * Must be called with the mutex held. The function will either
> > release the
> > + * mutex (if defering the release) or destroy it. The caller must
> > therefore not
> > + * reuse the buffer on return.
> > + */
> > +static void vc_sm_release_resource(struct vc_sm_buffer *buffer)
> > +{
> > + pr_debug("[%s]: buffer %p (name %s, size %zu), imported %u\n",
> > + __func__, buffer, buffer->name, buffer->size,
> > + buffer->imported);
> > +
> > + if (buffer->vc_handle) {
> > + /* We've sent the unmap request but not had the
> > response. */
> > + pr_debug("[%s]: Waiting for VPU unmap response on
> > %p\n",
> > + __func__, buffer);
> > + goto defer;
> > + }
> > + if (buffer->in_use) {
> > + /* dmabuf still in use - we await the release */
> > + pr_debug("[%s]: buffer %p is still in use\n", __func__,
> > buffer);
> > + goto defer;
> > + }
> > +
> > + /* Release the allocation (whether imported dmabuf or CMA
> > allocation) */
> > + if (buffer->imported) {
> > + if (buffer->import.dma_buf)
> > + dma_buf_put(buffer->import.dma_buf);
> > + else
> > + pr_err("%s: Imported dmabuf already been put
> > for buf %p\n",
> > + __func__, buffer);
> > + buffer->import.dma_buf = NULL;
> > + } else {
> > + dma_free_coherent(&sm_state->pdev->dev, buffer->size,
> > + buffer->cookie, buffer->dma_addr);
> > + }
> > +
> > + /* Free our buffer. Start by removing it from the list */
> > + mutex_lock(&sm_state->map_lock);
> > + list_del(&buffer->global_buffer_list);
> > + mutex_unlock(&sm_state->map_lock);
> > +
> > + pr_debug("%s: Release our allocation - done\n", __func__);
> > + mutex_unlock(&buffer->lock);
> > +
> > + mutex_destroy(&buffer->lock);
> > +
> > + kfree(buffer);
> > + return;
> > +
> > +defer:
> > + mutex_unlock(&buffer->lock);
> > +}
> > +
> > +/* Create support for private data tracking. */
> > +static struct vc_sm_privdata_t *vc_sm_cma_create_priv_data(pid_t id)
> > +{
> > + char alloc_name[32];
> > + struct vc_sm_privdata_t *file_data = NULL;
> > +
> > + /* Allocate private structure. */
> > + file_data = kzalloc(sizeof(*file_data), GFP_KERNEL);
> > +
> > + if (!file_data)
> > + return NULL;
> > +
> > + snprintf(alloc_name, sizeof(alloc_name), "%d", id);
> > +
> > + file_data->pid = id;
> > +
> > + return file_data;
> > +}
> > +
> > +/* Dma buf operations for use with our own allocations */
> > +
> > +static int vc_sm_dma_buf_attach(struct dma_buf *dmabuf,
> > + struct dma_buf_attachment *attachment)
> > +
> > +{
> > + struct vc_sm_dma_buf_attachment *a;
> > + struct sg_table *sgt;
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > + struct scatterlist *rd, *wr;
> > + int ret, i;
> > +
> > + a = kzalloc(sizeof(*a), GFP_KERNEL);
> > + if (!a)
> > + return -ENOMEM;
> > +
> > + pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf,
> > attachment);
> > +
> > + mutex_lock(&buf->lock);
> > +
> > + INIT_LIST_HEAD(&a->list);
> > +
> > + sgt = &a->sg_table;
> > +
> > + /* Copy the buf->base_sgt scatter list to the attachment, as we
> > can't
> > + * map the same scatter list to multiple attachments at the
> > same time.
> > + */
> > + ret = sg_alloc_table(sgt, buf->alloc.sg_table->orig_nents,
> > GFP_KERNEL);
> > + if (ret) {
> > + kfree(a);
> > + return -ENOMEM;
> > + }
> > +
> > + rd = buf->alloc.sg_table->sgl;
> > + wr = sgt->sgl;
> > + for (i = 0; i < sgt->orig_nents; ++i) {
> > + sg_set_page(wr, sg_page(rd), rd->length, rd->offset);
> > + rd = sg_next(rd);
> > + wr = sg_next(wr);
> > + }
> > +
> > + a->dma_dir = DMA_NONE;
> > + attachment->priv = a;
> > +
> > + list_add(&a->list, &buf->attachments);
> > + mutex_unlock(&buf->lock);
> > +
> > + return 0;
> > +}
> > +
> > +static void vc_sm_dma_buf_detach(struct dma_buf *dmabuf,
> > + struct dma_buf_attachment *attachment)
> > +{
> > + struct vc_sm_dma_buf_attachment *a = attachment->priv;
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > + struct sg_table *sgt;
> > +
> > + pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf,
> > attachment);
> > + if (!a)
> > + return;
> > +
> > + sgt = &a->sg_table;
> > +
> > + /* release the scatterlist cache */
> > + if (a->dma_dir != DMA_NONE)
> > + dma_unmap_sg(attachment->dev, sgt->sgl, sgt-
> > >orig_nents,
> > + a->dma_dir);
> > + sg_free_table(sgt);
> > +
> > + mutex_lock(&buf->lock);
> > + list_del(&a->list);
> > + mutex_unlock(&buf->lock);
> > +
> > + kfree(a);
> > +}
> > +
> > +static struct sg_table *vc_sm_map_dma_buf(struct dma_buf_attachment
> > *attachment,
> > + enum dma_data_direction
> > direction)
> > +{
> > + struct vc_sm_dma_buf_attachment *a = attachment->priv;
> > + /* stealing dmabuf mutex to serialize map/unmap operations */
> > + struct mutex *lock = &attachment->dmabuf->lock;
> > + struct sg_table *table;
> > +
> > + mutex_lock(lock);
> > + pr_debug("%s attachment %p\n", __func__, attachment);
> > + table = &a->sg_table;
> > +
> > + /* return previously mapped sg table */
> > + if (a->dma_dir == direction) {
> > + mutex_unlock(lock);
> > + return table;
> > + }
> > +
> > + /* release any previous cache */
> > + if (a->dma_dir != DMA_NONE) {
> > + dma_unmap_sg(attachment->dev, table->sgl, table-
> > >orig_nents,
> > + a->dma_dir);
> > + a->dma_dir = DMA_NONE;
> > + }
> > +
> > + /* mapping to the client with new direction */
> > + table->nents = dma_map_sg(attachment->dev, table->sgl,
> > + table->orig_nents, direction);
> > + if (!table->nents) {
> > + pr_err("failed to map scatterlist\n");
> > + mutex_unlock(lock);
> > + return ERR_PTR(-EIO);
> > + }
> > +
> > + a->dma_dir = direction;
> > + mutex_unlock(lock);
> > +
> > + pr_debug("%s attachment %p\n", __func__, attachment);
> > + return table;
> > +}
> > +
> > +static void vc_sm_unmap_dma_buf(struct dma_buf_attachment
> > *attachment,
> > + struct sg_table *table,
> > + enum dma_data_direction direction)
> > +{
> > + pr_debug("%s attachment %p\n", __func__, attachment);
> > + dma_unmap_sg(attachment->dev, table->sgl, table->nents,
> > direction);
> > +}
> > +
> > +static int vc_sm_dmabuf_mmap(struct dma_buf *dmabuf, struct
> > vm_area_struct *vma)
> > +{
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > + int ret;
> > +
> > + pr_debug("%s dmabuf %p, buf %p, vm_start %08lX\n", __func__,
> > dmabuf,
> > + buf, vma->vm_start);
> > +
> > + mutex_lock(&buf->lock);
> > +
> > + /* now map it to userspace */
> > + vma->vm_pgoff = 0;
> > +
> > + ret = dma_mmap_coherent(&sm_state->pdev->dev, vma, buf->cookie,
> > + buf->dma_addr, buf->size);
> > +
> > + if (ret) {
> > + pr_err("Remapping memory failed, error: %d\n", ret);
> > + return ret;
> > + }
> > +
> > + vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
> > +
> > + mutex_unlock(&buf->lock);
> > +
> > + if (ret)
> > + pr_err("%s: failure mapping buffer to userspace\n",
> > + __func__);
> > +
> > + return ret;
> > +}
> > +
> > +static void vc_sm_dma_buf_release(struct dma_buf *dmabuf)
> > +{
> > + struct vc_sm_buffer *buffer;
> > +
> > + if (!dmabuf)
> > + return;
> > +
> > + buffer = (struct vc_sm_buffer *)dmabuf->priv;
> > +
> > + mutex_lock(&buffer->lock);
> > +
> > + pr_debug("%s dmabuf %p, buffer %p\n", __func__, dmabuf,
> > buffer);
> > +
> > + buffer->in_use = 0;
> > +
> > + /* Unmap on the VPU */
> > + vc_sm_vpu_free(buffer);
> > + pr_debug("%s vpu_free done\n", __func__);
> > +
> > + /* Unmap our dma_buf object (the vc_sm_buffer remains until
> > released
> > + * on the VPU).
> > + */
> > + vc_sm_clean_up_dmabuf(buffer);
> > + pr_debug("%s clean_up dmabuf done\n", __func__);
> > +
> > + /* buffer->lock will be destroyed by vc_sm_release_resource if
> > finished
> > + * with, otherwise unlocked. Do NOT unlock here.
> > + */
> > + vc_sm_release_resource(buffer);
> > + pr_debug("%s done\n", __func__);
> > +}
> > +
> > +static int vc_sm_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> > + enum dma_data_direction
> > direction)
> > +{
> > + struct vc_sm_buffer *buf;
> > + struct vc_sm_dma_buf_attachment *a;
> > +
> > + if (!dmabuf)
> > + return -EFAULT;
> > +
> > + buf = dmabuf->priv;
> > + if (!buf)
> > + return -EFAULT;
> > +
> > + mutex_lock(&buf->lock);
> > +
> > + list_for_each_entry(a, &buf->attachments, list) {
> > + dma_sync_sg_for_cpu(a->dev, a->sg_table.sgl,
> > + a->sg_table.nents, direction);
> > + }
> > + mutex_unlock(&buf->lock);
> > +
> > + return 0;
> > +}
> > +
> > +static int vc_sm_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> > + enum dma_data_direction
> > direction)
> > +{
> > + struct vc_sm_buffer *buf;
> > + struct vc_sm_dma_buf_attachment *a;
> > +
> > + if (!dmabuf)
> > + return -EFAULT;
> > + buf = dmabuf->priv;
> > + if (!buf)
> > + return -EFAULT;
> > +
> > + mutex_lock(&buf->lock);
> > +
> > + list_for_each_entry(a, &buf->attachments, list) {
> > + dma_sync_sg_for_device(a->dev, a->sg_table.sgl,
> > + a->sg_table.nents, direction);
> > + }
> > + mutex_unlock(&buf->lock);
> > +
> > + return 0;
> > +}
> > +
> > +static const struct dma_buf_ops dma_buf_ops = {
> > + .map_dma_buf = vc_sm_map_dma_buf,
> > + .unmap_dma_buf = vc_sm_unmap_dma_buf,
> > + .mmap = vc_sm_dmabuf_mmap,
> > + .release = vc_sm_dma_buf_release,
> > + .attach = vc_sm_dma_buf_attach,
> > + .detach = vc_sm_dma_buf_detach,
> > + .begin_cpu_access = vc_sm_dma_buf_begin_cpu_access,
> > + .end_cpu_access = vc_sm_dma_buf_end_cpu_access,
> > +};
> > +
> > +/* Dma_buf operations for chaining through to an imported dma_buf */
> > +
> > +static
> > +int vc_sm_import_dma_buf_attach(struct dma_buf *dmabuf,
> > + struct dma_buf_attachment *attachment)
> > +{
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > +
> > + if (!buf->imported)
> > + return -EINVAL;
> > + return buf->import.dma_buf->ops->attach(buf->import.dma_buf,
> > + attachment);
> > +}
> > +
> > +static
> > +void vc_sm_import_dma_buf_detatch(struct dma_buf *dmabuf,
> > + struct dma_buf_attachment
> > *attachment)
> > +{
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > +
> > + if (!buf->imported)
> > + return;
> > + buf->import.dma_buf->ops->detach(buf->import.dma_buf,
> > attachment);
> > +}
> > +
> > +static
> > +struct sg_table *vc_sm_import_map_dma_buf(struct dma_buf_attachment
> > *attachment,
> > + enum dma_data_direction
> > direction)
> > +{
> > + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> > +
> > + if (!buf->imported)
> > + return NULL;
> > + return buf->import.dma_buf->ops->map_dma_buf(attachment,
> > + direction);
> > +}
> > +
> > +static
> > +void vc_sm_import_unmap_dma_buf(struct dma_buf_attachment
> > *attachment,
> > + struct sg_table *table,
> > + enum dma_data_direction direction)
> > +{
> > + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> > +
> > + if (!buf->imported)
> > + return;
> > + buf->import.dma_buf->ops->unmap_dma_buf(attachment, table,
> > direction);
> > +}
> > +
> > +static
> > +int vc_sm_import_dmabuf_mmap(struct dma_buf *dmabuf, struct
> > vm_area_struct *vma)
> > +{
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > +
> > + pr_debug("%s: mmap dma_buf %p, buf %p, imported db %p\n",
> > __func__,
> > + dmabuf, buf, buf->import.dma_buf);
> > + if (!buf->imported) {
> > + pr_err("%s: mmap dma_buf %p- not an imported buffer\n",
> > + __func__, dmabuf);
> > + return -EINVAL;
> > + }
> > + return buf->import.dma_buf->ops->mmap(buf->import.dma_buf,
> > vma);
> > +}
> > +
> > +static
> > +void vc_sm_import_dma_buf_release(struct dma_buf *dmabuf)
> > +{
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > +
> > + pr_debug("%s: Relasing dma_buf %p\n", __func__, dmabuf);
> > + mutex_lock(&buf->lock);
> > + if (!buf->imported)
> > + return;
> > +
> > + buf->in_use = 0;
> > +
> > + vc_sm_vpu_free(buf);
> > +
> > + vc_sm_release_resource(buf);
> > +}
> > +
> > +static
> > +int vc_sm_import_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> > + enum dma_data_direction
> > direction)
> > +{
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > +
> > + if (!buf->imported)
> > + return -EINVAL;
> > + return buf->import.dma_buf->ops->begin_cpu_access(buf-
> > >import.dma_buf,
> > + direction);
> > +}
> > +
> > +static
> > +int vc_sm_import_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> > + enum dma_data_direction
> > direction)
> > +{
> > + struct vc_sm_buffer *buf = dmabuf->priv;
> > +
> > + if (!buf->imported)
> > + return -EINVAL;
> > + return buf->import.dma_buf->ops->end_cpu_access(buf-
> > >import.dma_buf,
> > + direction);
> > +}
> > +
> > +static const struct dma_buf_ops dma_buf_import_ops = {
> > + .map_dma_buf = vc_sm_import_map_dma_buf,
> > + .unmap_dma_buf = vc_sm_import_unmap_dma_buf,
> > + .mmap = vc_sm_import_dmabuf_mmap,
> > + .release = vc_sm_import_dma_buf_release,
> > + .attach = vc_sm_import_dma_buf_attach,
> > + .detach = vc_sm_import_dma_buf_detatch,
> > + .begin_cpu_access = vc_sm_import_dma_buf_begin_cpu_access,
> > + .end_cpu_access = vc_sm_import_dma_buf_end_cpu_access,
> > +};
> > +
> > +/* Import a dma_buf to be shared with VC. */
> > +int
> > +vc_sm_cma_import_dmabuf_internal(struct vc_sm_privdata_t *private,
> > + struct dma_buf *dma_buf,
> > + int fd,
> > + struct dma_buf **imported_buf)
> > +{
> > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> > + struct vc_sm_buffer *buffer = NULL;
> > + struct vc_sm_import import = { };
> > + struct vc_sm_import_result result = { };
> > + struct dma_buf_attachment *attach = NULL;
> > + struct sg_table *sgt = NULL;
> > + dma_addr_t dma_addr;
> > + int ret = 0;
> > + int status;
> > +
> > + /* Setup our allocation parameters */
> > + pr_debug("%s: importing dma_buf %p/fd %d\n", __func__, dma_buf,
> > fd);
> > +
> > + if (fd < 0)
> > + get_dma_buf(dma_buf);
> > + else
> > + dma_buf = dma_buf_get(fd);
> > +
> > + if (!dma_buf)
> > + return -EINVAL;
> > +
> > + attach = dma_buf_attach(dma_buf, &sm_state->pdev->dev);
> > + if (IS_ERR(attach)) {
> > + ret = PTR_ERR(attach);
> > + goto error;
> > + }
> > +
> > + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> > + if (IS_ERR(sgt)) {
> > + ret = PTR_ERR(sgt);
> > + goto error;
> > + }
> > +
> > + /* Verify that the address block is contiguous */
> > + if (sgt->nents != 1) {
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + /* Allocate local buffer to track this allocation. */
> > + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> > + if (!buffer) {
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + import.type = VC_SM_ALLOC_NON_CACHED;
> > + dma_addr = sg_dma_address(sgt->sgl);
> > + import.addr = (u32)dma_addr;
> > + if ((import.addr & 0xC0000000) != 0xC0000000) {
> > + pr_err("%s: Expecting an uncached alias for dma_addr
> > %pad\n",
> > + __func__, &dma_addr);
> > + import.addr |= 0xC0000000;
> > + }
> > + import.size = sg_dma_len(sgt->sgl);
> > + import.allocator = current->tgid;
> > + import.kernel_id = get_kernel_id(buffer);
> > +
> > + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> > + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> > +
> > + pr_debug("[%s]: attempt to import \"%s\" data - type %u, addr
> > %pad, size %u.\n",
> > + __func__, import.name, import.type, &dma_addr,
> > import.size);
> > +
> > + /* Allocate the videocore buffer. */
> > + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import,
> > &result,
> > + &sm_state->int_trans_id);
> > + if (status == -EINTR) {
> > + pr_debug("[%s]: requesting import memory action restart
> > (trans_id: %u)\n",
> > + __func__, sm_state->int_trans_id);
> > + ret = -ERESTARTSYS;
> > + private->restart_sys = -EINTR;
> > + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> > + goto error;
> > + } else if (status || !result.res_handle) {
> > + pr_debug("[%s]: failed to import memory on videocore
> > (status: %u, trans_id: %u)\n",
> > + __func__, status, sm_state->int_trans_id);
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + mutex_init(&buffer->lock);
> > + INIT_LIST_HEAD(&buffer->attachments);
> > + memcpy(buffer->name, import.name,
> > + min(sizeof(buffer->name), sizeof(import.name) - 1));
> > +
> > + /* Keep track of the buffer we created. */
> > + buffer->private = private;
> > + buffer->vc_handle = result.res_handle;
> > + buffer->size = import.size;
> > + buffer->vpu_state = VPU_MAPPED;
> > +
> > + buffer->imported = 1;
> > + buffer->import.dma_buf = dma_buf;
> > +
> > + buffer->import.attach = attach;
> > + buffer->import.sgt = sgt;
> > + buffer->dma_addr = dma_addr;
> > + buffer->in_use = 1;
> > + buffer->kernel_id = import.kernel_id;
> > +
> > + /*
> > + * We're done - we need to export a new dmabuf chaining through
> > most
> > + * functions, but enabling us to release our own internal
> > references
> > + * here.
> > + */
> > + exp_info.ops = &dma_buf_import_ops;
> > + exp_info.size = import.size;
> > + exp_info.flags = O_RDWR;
> > + exp_info.priv = buffer;
> > +
> > + buffer->dma_buf = dma_buf_export(&exp_info);
> > + if (IS_ERR(buffer->dma_buf)) {
> > + ret = PTR_ERR(buffer->dma_buf);
> > + goto error;
> > + }
> > +
> > + vc_sm_add_resource(private, buffer);
> > +
> > + *imported_buf = buffer->dma_buf;
> > +
> > + return 0;
> > +
> > +error:
> > + if (result.res_handle) {
> > + struct vc_sm_free_t free = { result.res_handle, 0 };
> > +
> > + vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
> > + &sm_state->int_trans_id);
> > + }
> > + free_kernel_id(import.kernel_id);
> > + kfree(buffer);
> > + if (sgt)
> > + dma_buf_unmap_attachment(attach, sgt,
> > DMA_BIDIRECTIONAL);
> > + if (attach)
> > + dma_buf_detach(dma_buf, attach);
> > + dma_buf_put(dma_buf);
> > + return ret;
> > +}
> > +
> > +static int vc_sm_cma_vpu_alloc(u32 size, u32 align, const char
> > *name,
> > + u32 mem_handle, struct vc_sm_buffer
> > **ret_buffer)
> > +{
> > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> > + struct vc_sm_buffer *buffer = NULL;
> > + struct sg_table *sgt;
> > + int aligned_size;
> > + int ret = 0;
> > +
> > + /* Align to the user requested align */
> > + aligned_size = ALIGN(size, align);
> > + /* and then to a page boundary */
> > + aligned_size = PAGE_ALIGN(aligned_size);
> > +
> > + if (!aligned_size)
> > + return -EINVAL;
> > +
> > + /* Allocate local buffer to track this allocation. */
> > + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> > + if (!buffer)
> > + return -ENOMEM;
> > +
> > + mutex_init(&buffer->lock);
> > + /* Acquire the mutex as vc_sm_release_resource will release it
> > in the
> > + * error path.
> > + */
> > + mutex_lock(&buffer->lock);
> > +
> > + buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
> > + aligned_size, &buffer-
> > >dma_addr,
> > + GFP_KERNEL);
> > + if (!buffer->cookie) {
> > + pr_err("[%s]: dma_alloc_coherent alloc of %d bytes
> > failed\n",
> > + __func__, aligned_size);
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + pr_debug("[%s]: alloc of %d bytes success\n",
> > + __func__, aligned_size);
> > +
> > + sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
> > + if (!sgt) {
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer-
> > >cookie,
> > + buffer->dma_addr, buffer->size);
> > + if (ret < 0) {
> > + pr_err("failed to get scatterlist from DMA API\n");
> > + kfree(sgt);
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > + buffer->alloc.sg_table = sgt;
> > +
> > + INIT_LIST_HEAD(&buffer->attachments);
> > +
> > + memcpy(buffer->name, name,
> > + min(sizeof(buffer->name), strlen(name)));
> > +
> > + exp_info.ops = &dma_buf_ops;
> > + exp_info.size = aligned_size;
> > + exp_info.flags = O_RDWR;
> > + exp_info.priv = buffer;
> > +
> > + buffer->dma_buf = dma_buf_export(&exp_info);
> > + if (IS_ERR(buffer->dma_buf)) {
> > + ret = PTR_ERR(buffer->dma_buf);
> > + goto error;
> > + }
> > + buffer->dma_addr = (u32)sg_dma_address(buffer->alloc.sg_table-
> > >sgl);
> > + if ((buffer->dma_addr & 0xC0000000) != 0xC0000000) {
> > + pr_warn_once("%s: Expecting an uncached alias for
> > dma_addr %pad\n",
> > + __func__, &buffer->dma_addr);
> > + buffer->dma_addr |= 0xC0000000;
> > + }
> > + buffer->private = sm_state->vpu_allocs;
> > +
> > + buffer->vc_handle = mem_handle;
> > + buffer->vpu_state = VPU_MAPPED;
> > + buffer->vpu_allocated = 1;
> > + buffer->size = size;
> > + /*
> > + * Create an ID that will be passed along with our message so
> > + * that when we service the release reply, we can look up which
> > + * resource is being released.
> > + */
> > + buffer->kernel_id = get_kernel_id(buffer);
> > +
> > + vc_sm_add_resource(sm_state->vpu_allocs, buffer);
> > +
> > + mutex_unlock(&buffer->lock);
> > +
> > + *ret_buffer = buffer;
> > + return 0;
> > +error:
> > + if (buffer)
> > + vc_sm_release_resource(buffer);
> > + return ret;
> > +}
> > +
> > +static void
> > +vc_sm_vpu_event(struct sm_instance *instance, struct vc_sm_result_t
> > *reply,
> > + int reply_len)
> > +{
> > + switch (reply->trans_id & ~0x80000000) {
> > + case VC_SM_MSG_TYPE_CLIENT_VERSION:
> > + {
> > + /* Acknowledge that the firmware supports the version
> > command */
> > + pr_debug("%s: firmware acked version msg. Require
> > release cb\n",
> > + __func__);
> > + sm_state->require_released_callback = true;
> > + }
> > + break;
> > + case VC_SM_MSG_TYPE_RELEASED:
> > + {
> > + struct vc_sm_released *release = (struct vc_sm_released
> > *)reply;
> > + struct vc_sm_buffer *buffer =
> > + lookup_kernel_id(release-
> > >kernel_id);
> > + if (!buffer) {
> > + pr_err("%s: VC released a buffer that is
> > already released, kernel_id %d\n",
> > + __func__, release->kernel_id);
> > + break;
> > + }
> > + mutex_lock(&buffer->lock);
> > +
> > + pr_debug("%s: Released addr %08x, size %u, id %08x,
> > mem_handle %08x\n",
> > + __func__, release->addr, release->size,
> > + release->kernel_id, release->vc_handle);
> > +
> > + buffer->vc_handle = 0;
> > + buffer->vpu_state = VPU_NOT_MAPPED;
> > + free_kernel_id(release->kernel_id);
> > +
> > + if (buffer->vpu_allocated) {
> > + /* VPU allocation, so release the dmabuf which
> > will
> > + * trigger the clean up.
> > + */
> > + mutex_unlock(&buffer->lock);
> > + dma_buf_put(buffer->dma_buf);
> > + } else {
> > + vc_sm_release_resource(buffer);
> > + }
> > + }
> > + break;
> > + case VC_SM_MSG_TYPE_VC_MEM_REQUEST:
> > + {
> > + struct vc_sm_buffer *buffer = NULL;
> > + struct vc_sm_vc_mem_request *req =
> > + (struct vc_sm_vc_mem_request
> > *)reply;
> > + struct vc_sm_vc_mem_request_result reply;
> > + int ret;
> > +
> > + pr_debug("%s: Request %u bytes of memory, align %d name
> > %s, trans_id %08x\n",
> > + __func__, req->size, req->align, req->name,
> > + req->trans_id);
> > + ret = vc_sm_cma_vpu_alloc(req->size, req->align, req-
> > >name,
> > + req->vc_handle, &buffer);
> > +
> > + reply.trans_id = req->trans_id;
> > + if (!ret) {
> > + reply.addr = buffer->dma_addr;
> > + reply.kernel_id = buffer->kernel_id;
> > + pr_debug("%s: Allocated resource buffer %p,
> > addr %pad\n",
> > + __func__, buffer, &buffer->dma_addr);
> > + } else {
> > + pr_err("%s: Allocation failed size %u, name %s,
> > vc_handle %u\n",
> > + __func__, req->size, req->name, req-
> > >vc_handle);
> > + reply.addr = 0;
> > + reply.kernel_id = 0;
> > + }
> > + vc_sm_vchi_client_vc_mem_req_reply(sm_state->sm_handle,
> > &reply,
> > + &sm_state-
> > >int_trans_id);
> > + break;
> > + }
> > + break;
> > + default:
> > + pr_err("%s: Unknown vpu cmd %x\n", __func__, reply-
> > >trans_id);
> > + break;
> > + }
> > +}
> > +
> > +/* Userspace handling */
> > +/*
> > + * Open the device. Creates a private state to help track all
> > allocation
> > + * associated with this device.
> > + */
> > +static int vc_sm_cma_open(struct inode *inode, struct file *file)
> > +{
> > + /* Make sure the device was started properly. */
> > + if (!sm_state) {
> > + pr_err("[%s]: invalid device\n", __func__);
> > + return -EPERM;
> > + }
> > +
> > + file->private_data = vc_sm_cma_create_priv_data(current->tgid);
> > + if (!file->private_data) {
> > + pr_err("[%s]: failed to create data tracker\n",
> > __func__);
> > +
> > + return -ENOMEM;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +/*
> > + * Close the vcsm-cma device.
> > + * All allocations are file descriptors to the dmabuf objects, so we
> > will get
> > + * the clean up request on those as those are cleaned up.
> > + */
> > +static int vc_sm_cma_release(struct inode *inode, struct file *file)
> > +{
> > + struct vc_sm_privdata_t *file_data =
> > + (struct vc_sm_privdata_t *)file->private_data;
> > + int ret = 0;
> > +
> > + /* Make sure the device was started properly. */
> > + if (!sm_state || !file_data) {
> > + pr_err("[%s]: invalid device\n", __func__);
> > + ret = -EPERM;
> > + goto out;
> > + }
> > +
> > + pr_debug("[%s]: using private data %p\n", __func__, file_data);
> > +
> > + /* Terminate the private data. */
> > + kfree(file_data);
> > +
> > +out:
> > + return ret;
> > +}
> > +
> > +/*
> > + * Allocate a shared memory handle and block.
> > + * Allocation is from CMA, and then imported into the VPU mappings.
> > + */
> > +int vc_sm_cma_ioctl_alloc(struct vc_sm_privdata_t *private,
> > + struct vc_sm_cma_ioctl_alloc *ioparam)
> > +{
> > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> > + struct vc_sm_buffer *buffer = NULL;
> > + struct vc_sm_import import = { 0 };
> > + struct vc_sm_import_result result = { 0 };
> > + struct dma_buf *dmabuf = NULL;
> > + struct sg_table *sgt;
> > + int aligned_size;
> > + int ret = 0;
> > + int status;
> > + int fd = -1;
> > +
> > + aligned_size = PAGE_ALIGN(ioparam->size);
> > +
> > + if (!aligned_size)
> > + return -EINVAL;
> > +
> > + /* Allocate local buffer to track this allocation. */
> > + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> > + if (!buffer) {
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
> > + aligned_size,
> > + &buffer->dma_addr,
> > + GFP_KERNEL);
> > + if (!buffer->cookie) {
> > + pr_err("[%s]: dma_alloc_coherent alloc of %d bytes
> > failed\n",
> > + __func__, aligned_size);
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + import.type = VC_SM_ALLOC_NON_CACHED;
> > + import.allocator = current->tgid;
> > +
> > + if (*ioparam->name)
> > + memcpy(import.name, ioparam->name, sizeof(import.name)
> > - 1);
> > + else
> > + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> > + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> > +
> > + mutex_init(&buffer->lock);
> > + INIT_LIST_HEAD(&buffer->attachments);
> > + memcpy(buffer->name, import.name,
> > + min(sizeof(buffer->name), sizeof(import.name) - 1));
> > +
> > + exp_info.ops = &dma_buf_ops;
> > + exp_info.size = aligned_size;
> > + exp_info.flags = O_RDWR;
> > + exp_info.priv = buffer;
> > +
> > + dmabuf = dma_buf_export(&exp_info);
> > + if (IS_ERR(dmabuf)) {
> > + ret = PTR_ERR(dmabuf);
> > + goto error;
> > + }
> > + buffer->dma_buf = dmabuf;
> > +
> > + import.addr = buffer->dma_addr;
> > + import.size = aligned_size;
> > + import.kernel_id = get_kernel_id(buffer);
> > +
> > + /* Wrap it into a videocore buffer. */
> > + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import,
> > &result,
> > + &sm_state->int_trans_id);
> > + if (status == -EINTR) {
> > + pr_debug("[%s]: requesting import memory action restart
> > (trans_id: %u)\n",
> > + __func__, sm_state->int_trans_id);
> > + ret = -ERESTARTSYS;
> > + private->restart_sys = -EINTR;
> > + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> > + goto error;
> > + } else if (status || !result.res_handle) {
> > + pr_err("[%s]: failed to import memory on videocore
> > (status: %u, trans_id: %u)\n",
> > + __func__, status, sm_state->int_trans_id);
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + /* Keep track of the buffer we created. */
> > + buffer->private = private;
> > + buffer->vc_handle = result.res_handle;
> > + buffer->size = import.size;
> > + buffer->vpu_state = VPU_MAPPED;
> > + buffer->kernel_id = import.kernel_id;
> > +
> > + sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
> > + if (!sgt) {
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer-
> > >cookie,
> > + buffer->dma_addr, buffer->size);
> > + if (ret < 0) {
> > + /* FIXME: error handling */
> > + pr_err("failed to get scatterlist from DMA API\n");
> > + kfree(sgt);
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > + buffer->alloc.sg_table = sgt;
> > +
> > + fd = dma_buf_fd(dmabuf, O_CLOEXEC);
> > + if (fd < 0)
> > + goto error;
> > +
> > + vc_sm_add_resource(private, buffer);
> > +
> > + pr_debug("[%s]: Added resource as fd %d, buffer %p, private %p,
> > dma_addr %pad\n",
> > + __func__, fd, buffer, private, &buffer->dma_addr);
> > +
> > + /* We're done */
> > + ioparam->handle = fd;
> > + ioparam->vc_handle = buffer->vc_handle;
> > + ioparam->dma_addr = buffer->dma_addr;
> > + return 0;
> > +
> > +error:
> > + pr_err("[%s]: something failed - cleanup. ret %d\n", __func__,
> > ret);
> > +
> > + if (dmabuf) {
> > + /* dmabuf has been exported, therefore allow dmabuf
> > cleanup to
> > + * deal with this
> > + */
> > + dma_buf_put(dmabuf);
> > + } else {
> > + /* No dmabuf, therefore just free the buffer here */
> > + if (buffer->cookie)
> > + dma_free_coherent(&sm_state->pdev->dev, buffer-
> > >size,
> > + buffer->cookie, buffer-
> > >dma_addr);
> > + kfree(buffer);
> > + }
> > + return ret;
> > +}
> > +
> > +#ifndef CONFIG_ARM64
> > +/* Converts VCSM_CACHE_OP_* to an operating function. */
> > +static void (*cache_op_to_func(const unsigned int cache_op))
> > + (const void*, const
> > void*)
> > +{
> > + switch (cache_op) {
> > + case VC_SM_CACHE_OP_NOP:
> > + return NULL;
> > +
> > + case VC_SM_CACHE_OP_INV:
> > + case VC_SM_CACHE_OP_CLEAN:
> > + case VC_SM_CACHE_OP_FLUSH:
> > + return dmac_flush_range;
> > +
> > + default:
> > + pr_err("[%s]: Invalid cache_op: 0x%08x\n", __func__,
> > cache_op);
> > + return NULL;
> > + }
> > +}
> > +
> > +/*
> > + * Clean/invalid/flush cache of which buffer is already pinned (i.e.
> > accessed).
> > + */
> > +static int clean_invalid_contig_2d(const void __user *addr,
> > + const size_t block_count,
> > + const size_t block_size,
> > + const size_t stride,
> > + const unsigned int cache_op)
> > +{
> > + size_t i;
> > + void (*op_fn)(const void *start, const void *end);
> > +
> > + if (!block_size) {
> > + pr_err("[%s]: size cannot be 0\n", __func__);
> > + return -EINVAL;
> > + }
> > +
> > + op_fn = cache_op_to_func(cache_op);
> > + if (!op_fn)
> > + return -EINVAL;
> > +
> > + for (i = 0; i < block_count; i ++, addr += stride)
> > + op_fn(addr, addr + block_size);
> > +
> > + return 0;
> > +}
> > +
> > +static int vc_sm_cma_clean_invalid2(unsigned int cmdnr, unsigned
> > long arg)
> > +{
> > + struct vc_sm_cma_ioctl_clean_invalid2 ioparam;
> > + struct vc_sm_cma_ioctl_clean_invalid_block *block = NULL;
> > + int i, ret = 0;
> > +
> > + /* Get parameter data. */
> > + if (copy_from_user(&ioparam, (void *)arg, sizeof(ioparam))) {
> > + pr_err("[%s]: failed to copy-from-user header for cmd
> > %x\n",
> > + __func__, cmdnr);
> > + return -EFAULT;
> > + }
> > + block = kmalloc(ioparam.op_count * sizeof(*block), GFP_KERNEL);
> > + if (!block)
> > + return -EFAULT;
> > +
> > + if (copy_from_user(block, (void *)(arg + sizeof(ioparam)),
> > + ioparam.op_count * sizeof(*block)) != 0) {
> > + pr_err("[%s]: failed to copy-from-user payload for cmd
> > %x\n",
> > + __func__, cmdnr);
> > + ret = -EFAULT;
> > + goto out;
> > + }
> > +
> > + for (i = 0; i < ioparam.op_count; i++) {
> > + const struct vc_sm_cma_ioctl_clean_invalid_block *
> > const op =
> > + block +
> > i;
> > +
> > + if (op->invalidate_mode == VC_SM_CACHE_OP_NOP)
> > + continue;
> > +
> > + ret = clean_invalid_contig_2d((void __user *)op-
> > >start_address,
> > + op->block_count, op-
> > >block_size,
> > + op->inter_block_stride,
> > + op->invalidate_mode);
> > + if (ret)
> > + break;
> > + }
> > +out:
> > + kfree(block);
> > +
> > + return ret;
> > +}
> > +#endif
> > +
> > +static long vc_sm_cma_ioctl(struct file *file, unsigned int cmd,
> > + unsigned long arg)
> > +{
> > + int ret = 0;
> > + unsigned int cmdnr = _IOC_NR(cmd);
> > + struct vc_sm_privdata_t *file_data =
> > + (struct vc_sm_privdata_t *)file->private_data;
> > +
> > + /* Validate we can work with this device. */
> > + if (!sm_state || !file_data) {
> > + pr_err("[%s]: invalid device\n", __func__);
> > + return -EPERM;
> > + }
> > +
> > + /* Action is a re-post of a previously interrupted action? */
> > + if (file_data->restart_sys == -EINTR) {
> > + struct vc_sm_action_clean_t action_clean;
> > +
> > + pr_debug("[%s]: clean up of action %u (trans_id: %u)
> > following EINTR\n",
> > + __func__, file_data->int_action,
> > + file_data->int_trans_id);
> > +
> > + action_clean.res_action = file_data->int_action;
> > + action_clean.action_trans_id = file_data->int_trans_id;
> > +
> > + file_data->restart_sys = 0;
> > + }
> > +
> > + /* Now process the command. */
> > + switch (cmdnr) {
> > + /* New memory allocation.
> > + */
> > + case VC_SM_CMA_CMD_ALLOC:
> > + {
> > + struct vc_sm_cma_ioctl_alloc ioparam;
> > +
> > + /* Get the parameter data. */
> > + if (copy_from_user
> > + (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
> > + pr_err("[%s]: failed to copy-from-user for cmd
> > %x\n",
> > + __func__, cmdnr);
> > + ret = -EFAULT;
> > + break;
> > + }
> > +
> > + ret = vc_sm_cma_ioctl_alloc(file_data, &ioparam);
> > + if (!ret &&
> > + (copy_to_user((void *)arg, &ioparam,
> > + sizeof(ioparam)) != 0)) {
> > + /* FIXME: Release allocation */
> > + pr_err("[%s]: failed to copy-to-user for cmd
> > %x\n",
> > + __func__, cmdnr);
> > + ret = -EFAULT;
> > + }
> > + break;
> > + }
> > +
> > + case VC_SM_CMA_CMD_IMPORT_DMABUF:
> > + {
> > + struct vc_sm_cma_ioctl_import_dmabuf ioparam;
> > + struct dma_buf *new_dmabuf;
> > +
> > + /* Get the parameter data. */
> > + if (copy_from_user
> > + (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
> > + pr_err("[%s]: failed to copy-from-user for cmd
> > %x\n",
> > + __func__, cmdnr);
> > + ret = -EFAULT;
> > + break;
> > + }
> > +
> > + ret = vc_sm_cma_import_dmabuf_internal(file_data,
> > + NULL,
> > + ioparam.dmabuf_f
> > d,
> > + &new_dmabuf);
> > +
> > + if (!ret) {
> > + struct vc_sm_buffer *buf = new_dmabuf->priv;
> > +
> > + ioparam.size = buf->size;
> > + ioparam.handle = dma_buf_fd(new_dmabuf,
> > + O_CLOEXEC);
> > + ioparam.vc_handle = buf->vc_handle;
> > + ioparam.dma_addr = buf->dma_addr;
> > +
> > + if (ioparam.handle < 0 ||
> > + (copy_to_user((void *)arg, &ioparam,
> > + sizeof(ioparam)) != 0)) {
> > + dma_buf_put(new_dmabuf);
> > + /* FIXME: Release allocation */
> > + ret = -EFAULT;
> > + }
> > + }
> > + break;
> > + }
> > +
> > +#ifndef CONFIG_ARM64
> > + /*
> > + * Flush/Invalidate the cache for a given mapping.
> > + * Blocks must be pinned (i.e. accessed) before this call.
> > + */
> > + case VC_SM_CMA_CMD_CLEAN_INVALID2:
> > + ret = vc_sm_cma_clean_invalid2(cmdnr, arg);
> > + break;
> > +#endif
> > +
> > + default:
> > + pr_debug("[%s]: cmd %x tgid %u, owner %u\n", __func__,
> > cmdnr,
> > + current->tgid, file_data->pid);
> > +
> > + ret = -EINVAL;
> > + break;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +#ifdef CONFIG_COMPAT
> > +struct vc_sm_cma_ioctl_clean_invalid2_32 {
> > + u32 op_count;
> > + struct vc_sm_cma_ioctl_clean_invalid_block_32 {
> > + u16 invalidate_mode;
> > + u16 block_count;
> > + compat_uptr_t start_address;
> > + u32 block_size;
> > + u32 inter_block_stride;
> > + } s[0];
> > +};
> > +
> > +#define VC_SM_CMA_CMD_CLEAN_INVALID2_32\
> > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
> > + struct vc_sm_cma_ioctl_clean_invalid2_32)
> > +
> > +static long vc_sm_cma_compat_ioctl(struct file *file, unsigned int
> > cmd,
> > + unsigned long arg)
> > +{
> > + switch (cmd) {
> > + case VC_SM_CMA_CMD_CLEAN_INVALID2_32:
> > + /* FIXME */
> > + return -EINVAL;
> > +
> > + default:
> > + return vc_sm_cma_ioctl(file, cmd, arg);
> > + }
> > +}
> > +#endif
> > +
> > +/* Device operations that we managed in this driver. */
> > +static const struct file_operations vc_sm_ops = {
> > + .owner = THIS_MODULE,
> > + .unlocked_ioctl = vc_sm_cma_ioctl,
> > +#ifdef CONFIG_COMPAT
> > + .compat_ioctl = vc_sm_cma_compat_ioctl,
> > +#endif
> > + .open = vc_sm_cma_open,
> > + .release = vc_sm_cma_release,
> > +};
> > +
> > +/* Driver load/unload functions */
> > +/* Videocore connected. */
> > +static void vc_sm_connected_init(void)
> > +{
> > + int ret;
> > + struct vchi_instance_handle *vchi_instance;
> > + struct vc_sm_version version;
> > + struct vc_sm_result_t version_result;
> > +
> > + pr_info("[%s]: start\n", __func__);
> > +
> > + /*
> > + * Initialize and create a VCHI connection for the shared
> > memory service
> > + * running on videocore.
> > + */
> > + ret = vchi_initialise(&vchi_instance);
> > + if (ret) {
> > + pr_err("[%s]: failed to initialise VCHI instance
> > (ret=%d)\n",
> > + __func__, ret);
> > +
> > + return;
> > + }
> > +
> > + ret = vchi_connect(vchi_instance);
> > + if (ret) {
> > + pr_err("[%s]: failed to connect VCHI instance
> > (ret=%d)\n",
> > + __func__, ret);
> > +
> > + return;
> > + }
> > +
> > + /* Initialize an instance of the shared memory service. */
> > + sm_state->sm_handle = vc_sm_cma_vchi_init(vchi_instance, 1,
> > + vc_sm_vpu_event);
> > + if (!sm_state->sm_handle) {
> > + pr_err("[%s]: failed to initialize shared memory
> > service\n",
> > + __func__);
> > +
> > + return;
> > + }
> > +
> > + /* Create a debug fs directory entry (root). */
> > + sm_state->dir_root = debugfs_create_dir(VC_SM_DIR_ROOT_NAME,
> > NULL);
> > +
> > + sm_state->dir_state.show = &vc_sm_cma_global_state_show;
> > + sm_state->dir_state.dir_entry =
> > + debugfs_create_file(VC_SM_STATE, 0444, sm_state-
> > >dir_root,
> > + &sm_state->dir_state,
> > + &vc_sm_cma_debug_fs_fops);
> > +
> > + INIT_LIST_HEAD(&sm_state->buffer_list);
> > +
> > + /* Create a shared memory device. */
> > + sm_state->misc_dev.minor = MISC_DYNAMIC_MINOR;
> > + sm_state->misc_dev.name = DEVICE_NAME;
> > + sm_state->misc_dev.fops = &vc_sm_ops;
> > + sm_state->misc_dev.parent = NULL;
> > + /* Temporarily set as 666 until udev rules have been sorted */
> > + sm_state->misc_dev.mode = 0666;
> > + ret = misc_register(&sm_state->misc_dev);
> > + if (ret) {
> > + pr_err("vcsm-cma: failed to register misc device.\n");
> > + goto err_remove_debugfs;
> > + }
> > +
> > + sm_state->data_knl = vc_sm_cma_create_priv_data(0);
> > + if (!sm_state->data_knl) {
> > + pr_err("[%s]: failed to create kernel private data
> > tracker\n",
> > + __func__);
> > + goto err_remove_misc_dev;
> > + }
> > +
> > + version.version = 2;
> > + ret = vc_sm_cma_vchi_client_version(sm_state->sm_handle,
> > &version,
> > + &version_result,
> > + &sm_state->int_trans_id);
> > + if (ret) {
> > + pr_err("[%s]: Failed to send version request %d\n",
> > __func__,
> > + ret);
> > + }
> > +
> > + /* Done! */
> > + sm_inited = 1;
> > + pr_info("[%s]: installed successfully\n", __func__);
> > + return;
> > +
> > +err_remove_misc_dev:
> > + misc_deregister(&sm_state->misc_dev);
> > +err_remove_debugfs:
> > + debugfs_remove_recursive(sm_state->dir_root);
> > + vc_sm_cma_vchi_stop(&sm_state->sm_handle);
> > +}
> > +
> > +/* Driver loading. */
> > +static int bcm2835_vc_sm_cma_probe(struct platform_device *pdev)
> > +{
> > + pr_info("%s: Videocore shared memory driver\n", __func__);
> > +
> > + sm_state = devm_kzalloc(&pdev->dev, sizeof(*sm_state),
> > GFP_KERNEL);
> > + if (!sm_state)
> > + return -ENOMEM;
> > + sm_state->pdev = pdev;
> > + mutex_init(&sm_state->map_lock);
> > +
> > + spin_lock_init(&sm_state->kernelid_map_lock);
> > + idr_init_base(&sm_state->kernelid_map, 1);
> > +
> > + pdev->dev.dma_parms = devm_kzalloc(&pdev->dev,
> > + sizeof(*pdev-
> > >dev.dma_parms),
> > + GFP_KERNEL);
> > + /* dma_set_max_seg_size checks if dma_parms is NULL. */
> > + dma_set_max_seg_size(&pdev->dev, 0x3FFFFFFF);
> > +
> > + vchiq_add_connected_callback(vc_sm_connected_init);
> > + return 0;
> > +}
> > +
> > +/* Driver unloading. */
> > +static int bcm2835_vc_sm_cma_remove(struct platform_device *pdev)
> > +{
> > + pr_debug("[%s]: start\n", __func__);
> > + if (sm_inited) {
> > + misc_deregister(&sm_state->misc_dev);
> > +
> > + /* Remove all proc entries. */
> > + debugfs_remove_recursive(sm_state->dir_root);
> > +
> > + /* Stop the videocore shared memory service. */
> > + vc_sm_cma_vchi_stop(&sm_state->sm_handle);
> > + }
> > +
> > + if (sm_state) {
> > + idr_destroy(&sm_state->kernelid_map);
> > +
> > + /* Free the memory for the state structure. */
> > + mutex_destroy(&sm_state->map_lock);
> > + }
> > +
> > + pr_debug("[%s]: end\n", __func__);
> > + return 0;
> > +}
> > +
> > +/* Kernel API calls */
> > +/* Get an internal resource handle mapped from the external one. */
> > +int vc_sm_cma_int_handle(void *handle)
> > +{
> > + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> > + struct vc_sm_buffer *buf;
> > +
> > + /* Validate we can work with this device. */
> > + if (!sm_state || !handle) {
> > + pr_err("[%s]: invalid input\n", __func__);
> > + return 0;
> > + }
> > +
> > + buf = (struct vc_sm_buffer *)dma_buf->priv;
> > + return buf->vc_handle;
> > +}
> > +EXPORT_SYMBOL_GPL(vc_sm_cma_int_handle);
> > +
> > +/* Free a previously allocated shared memory handle and block. */
> > +int vc_sm_cma_free(void *handle)
> > +{
> > + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> > +
> > + /* Validate we can work with this device. */
> > + if (!sm_state || !handle) {
> > + pr_err("[%s]: invalid input\n", __func__);
> > + return -EPERM;
> > + }
> > +
> > + pr_debug("%s: handle %p/dmabuf %p\n", __func__, handle,
> > dma_buf);
> > +
> > + dma_buf_put(dma_buf);
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(vc_sm_cma_free);
> > +
> > +/* Import a dmabuf to be shared with VC. */
> > +int vc_sm_cma_import_dmabuf(struct dma_buf *src_dmabuf, void
> > **handle)
> > +{
> > + struct dma_buf *new_dma_buf;
> > + struct vc_sm_buffer *buf;
> > + int ret;
> > +
> > + /* Validate we can work with this device. */
> > + if (!sm_state || !src_dmabuf || !handle) {
> > + pr_err("[%s]: invalid input\n", __func__);
> > + return -EPERM;
> > + }
> > +
> > + ret = vc_sm_cma_import_dmabuf_internal(sm_state->data_knl,
> > src_dmabuf,
> > + -1, &new_dma_buf);
> > +
> > + if (!ret) {
> > + pr_debug("%s: imported to ptr %p\n", __func__,
> > new_dma_buf);
> > + buf = (struct vc_sm_buffer *)new_dma_buf->priv;
> > +
> > + /* Assign valid handle at this time.*/
> > + *handle = new_dma_buf;
> > + } else {
> > + /*
> > + * succeeded in importing the dma_buf, but then
> > + * failed to look it up again. How?
> > + * Release the fd again.
> > + */
> > + pr_err("%s: imported vc_sm_cma_get_buffer failed %d\n",
> > + __func__, ret);
> > + }
> > +
> > + return ret;
> > +}
> > +EXPORT_SYMBOL_GPL(vc_sm_cma_import_dmabuf);
> > +
> > +static struct platform_driver bcm2835_vcsm_cma_driver = {
> > + .probe = bcm2835_vc_sm_cma_probe,
> > + .remove = bcm2835_vc_sm_cma_remove,
> > + .driver = {
> > + .name = DEVICE_NAME,
> > + .owner = THIS_MODULE,
> > + },
> > +};
> > +
> > +module_platform_driver(bcm2835_vcsm_cma_driver);
> > +
> > +MODULE_AUTHOR("Dave Stevenson");
> > +MODULE_DESCRIPTION("VideoCore CMA Shared Memory Driver");
> > +MODULE_LICENSE("GPL v2");
> > +MODULE_ALIAS("platform:vcsm-cma");
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > new file mode 100644
> > index 000000000000..f1c7b95b14ce
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > @@ -0,0 +1,84 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +/*
> > + * VideoCore Shared Memory driver using CMA.
> > + *
> > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > + *
> > + */
> > +
> > +#ifndef VC_SM_H
> > +#define VC_SM_H
> > +
> > +#include <linux/device.h>
> > +#include <linux/dma-direction.h>
> > +#include <linux/kref.h>
> > +#include <linux/mm_types.h>
> > +#include <linux/mutex.h>
> > +#include <linux/rbtree.h>
> > +#include <linux/sched.h>
> > +#include <linux/shrinker.h>
> > +#include <linux/types.h>
> > +#include <linux/miscdevice.h>
> > +
> > +#define VC_SM_MAX_NAME_LEN 32
> > +
> > +enum vc_sm_vpu_mapping_state {
> > + VPU_NOT_MAPPED,
> > + VPU_MAPPED,
> > + VPU_UNMAPPING
> > +};
> > +
> > +struct vc_sm_alloc_data {
> > + unsigned long num_pages;
> > + void *priv_virt;
> > + struct sg_table *sg_table;
> > +};
> > +
> > +struct vc_sm_imported {
> > + struct dma_buf *dma_buf;
> > + struct dma_buf_attachment *attach;
> > + struct sg_table *sgt;
> > +};
> > +
> > +struct vc_sm_buffer {
> > + struct list_head global_buffer_list; /* Global list of
> > buffers. */
> > +
> > + /* Index in the kernel_id idr so that we can find the
> > + * mmal_msg_context again when servicing the VCHI reply.
> > + */
> > + int kernel_id;
> > +
> > + size_t size;
> > +
> > + /* Lock over all the following state for this buffer */
> > + struct mutex lock;
> > + struct list_head attachments;
> > +
> > + char name[VC_SM_MAX_NAME_LEN];
> > +
> > + int in_use:1; /* Kernel is still using this resource */
> > + int imported:1; /* Imported dmabuf */
> > +
> > + enum vc_sm_vpu_mapping_state vpu_state;
> > + u32 vc_handle; /* VideoCore handle for this buffer */
> > + int vpu_allocated; /*
> > + * The VPU made this allocation.
> > Release the
> > + * local dma_buf when the VPU releases
> > the
> > + * resource.
> > + */
> > +
> > + /* DMABUF related fields */
> > + struct dma_buf *dma_buf;
> > + dma_addr_t dma_addr;
> > + void *cookie;
> > +
> > + struct vc_sm_privdata_t *private;
> > +
> > + union {
> > + struct vc_sm_alloc_data alloc;
> > + struct vc_sm_imported import;
> > + };
> > +};
> > +
> > +#endif
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> > new file mode 100644
> > index 000000000000..6a203c60bf7f
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> > @@ -0,0 +1,505 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * VideoCore Shared Memory CMA allocator
> > + *
> > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> > + *
> > + * Based on vmcs_sm driver from Broadcom Corporation.
> > + *
> > + */
> > +
> > +/* ---- Include Files ----------------------------------------------
> > ------- */
> > +#include <linux/completion.h>
> > +#include <linux/kernel.h>
> > +#include <linux/kthread.h>
> > +#include <linux/list.h>
> > +#include <linux/mutex.h>
> > +#include <linux/semaphore.h>
> > +#include <linux/slab.h>
> > +#include <linux/types.h>
> > +
> > +#include "vc_sm_cma_vchi.h"
> > +
> > +#define VC_SM_VER 1
> > +#define VC_SM_MIN_VER 0
> > +
> > +/* ---- Private Constants and Types --------------------------------
> > ------ */
> > +
> > +/* Command blocks come from a pool */
> > +#define SM_MAX_NUM_CMD_RSP_BLKS 32
> > +
> > +struct sm_cmd_rsp_blk {
> > + struct list_head head; /* To create lists */
> > + /* To be signaled when the response is there */
> > + struct completion cmplt;
> > +
> > + u32 id;
> > + u16 length;
> > +
> > + u8 msg[VC_SM_MAX_MSG_LEN];
> > +
> > + uint32_t wait:1;
> > + uint32_t sent:1;
> > + uint32_t alloc:1;
> > +
> > +};
> > +
> > +struct sm_instance {
> > + u32 num_connections;
> > + struct vchi_service_handle
> > *vchi_handle[VCHI_MAX_NUM_CONNECTIONS];
> > + struct task_struct *io_thread;
> > + struct completion io_cmplt;
> > +
> > + vpu_event_cb vpu_event;
> > +
> > + /* Mutex over the following lists */
> > + struct mutex lock;
> > + u32 trans_id;
> > + struct list_head cmd_list;
> > + struct list_head rsp_list;
> > + struct list_head dead_list;
> > +
> > + struct sm_cmd_rsp_blk free_blk[SM_MAX_NUM_CMD_RSP_BLKS];
> > +
> > + /* Mutex over the free_list */
> > + struct mutex free_lock;
> > + struct list_head free_list;
> > +
> > + struct semaphore free_sema;
> > +
> > +};
> > +
> > +/* ---- Private Variables ------------------------------------------
> > ------ */
> > +
> > +/* ---- Private Function Prototypes --------------------------------
> > ------ */
> > +
> > +/* ---- Private Functions ------------------------------------------
> > ------ */
> > +static int
> > +bcm2835_vchi_msg_queue(struct vchi_service_handle *handle,
> > + void *data,
> > + unsigned int size)
> > +{
> > + return vchi_queue_kernel_message(handle,
> > + data,
> > + size);
> > +}
> > +
> > +static struct
> > +sm_cmd_rsp_blk *vc_vchi_cmd_create(struct sm_instance *instance,
> > + enum vc_sm_msg_type id, void *msg,
> > + u32 size, int wait)
> > +{
> > + struct sm_cmd_rsp_blk *blk;
> > + struct vc_sm_msg_hdr_t *hdr;
> > +
> > + if (down_interruptible(&instance->free_sema)) {
> > + blk = kmalloc(sizeof(*blk), GFP_KERNEL);
> > + if (!blk)
> > + return NULL;
> > +
> > + blk->alloc = 1;
> > + init_completion(&blk->cmplt);
> > + } else {
> > + mutex_lock(&instance->free_lock);
> > + blk =
> > + list_first_entry(&instance->free_list,
> > + struct sm_cmd_rsp_blk, head);
> > + list_del(&blk->head);
> > + mutex_unlock(&instance->free_lock);
> > + }
> > +
> > + blk->sent = 0;
> > + blk->wait = wait;
> > + blk->length = sizeof(*hdr) + size;
> > +
> > + hdr = (struct vc_sm_msg_hdr_t *)blk->msg;
> > + hdr->type = id;
> > + mutex_lock(&instance->lock);
> > + instance->trans_id++;
> > + /*
> > + * Retain the top bit for identifying asynchronous events, or
> > VPU cmds.
> > + */
> > + instance->trans_id &= ~0x80000000;
> > + hdr->trans_id = instance->trans_id;
> > + blk->id = instance->trans_id;
> > + mutex_unlock(&instance->lock);
> > +
> > + if (size)
> > + memcpy(hdr->body, msg, size);
> > +
> > + return blk;
> > +}
> > +
> > +static void
> > +vc_vchi_cmd_delete(struct sm_instance *instance, struct
> > sm_cmd_rsp_blk *blk)
> > +{
> > + if (blk->alloc) {
> > + kfree(blk);
> > + return;
> > + }
> > +
> > + mutex_lock(&instance->free_lock);
> > + list_add(&blk->head, &instance->free_list);
> > + mutex_unlock(&instance->free_lock);
> > + up(&instance->free_sema);
> > +}
> > +
> > +static void vc_sm_cma_vchi_rx_ack(struct sm_instance *instance,
> > + struct sm_cmd_rsp_blk *cmd,
> > + struct vc_sm_result_t *reply,
> > + u32 reply_len)
> > +{
> > + mutex_lock(&instance->lock);
> > + list_for_each_entry(cmd,
> > + &instance->rsp_list,
> > + head) {
> > + if (cmd->id == reply->trans_id)
> > + break;
> > + }
> > + mutex_unlock(&instance->lock);
> > +
> > + if (&cmd->head == &instance->rsp_list) {
> > + //pr_debug("%s: received response %u, throw away...",
> > + pr_err("%s: received response %u, throw away...",
> > + __func__,
> > + reply->trans_id);
> > + } else if (reply_len > sizeof(cmd->msg)) {
> > + pr_err("%s: reply too big (%u) %u, throw away...",
> > + __func__, reply_len,
> > + reply->trans_id);
> > + } else {
> > + memcpy(cmd->msg, reply,
> > + reply_len);
> > + complete(&cmd->cmplt);
> > + }
> > +}
> > +
> > +static int vc_sm_cma_vchi_videocore_io(void *arg)
> > +{
> > + struct sm_instance *instance = arg;
> > + struct sm_cmd_rsp_blk *cmd = NULL, *cmd_tmp;
> > + struct vc_sm_result_t *reply;
> > + u32 reply_len;
> > + s32 status;
> > + int svc_use = 1;
> > +
> > + while (1) {
> > + if (svc_use)
> > + vchi_service_release(instance->vchi_handle[0]);
> > + svc_use = 0;
> > +
> > + if (wait_for_completion_interruptible(&instance-
> > >io_cmplt))
> > + continue;
> > +
> > + vchi_service_use(instance->vchi_handle[0]);
> > + svc_use = 1;
> > +
> > + do {
> > + /*
> > + * Get new command and move it to response list
> > + */
> > + mutex_lock(&instance->lock);
> > + if (list_empty(&instance->cmd_list)) {
> > + /* no more commands to process */
> > + mutex_unlock(&instance->lock);
> > + break;
> > + }
> > + cmd = list_first_entry(&instance->cmd_list,
> > + struct sm_cmd_rsp_blk,
> > head);
> > + list_move(&cmd->head, &instance->rsp_list);
> > + cmd->sent = 1;
> > + mutex_unlock(&instance->lock);
> > +
> > + /* Send the command */
> > + status =
> > + bcm2835_vchi_msg_queue(instance-
> > >vchi_handle[0],
> > + cmd->msg, cmd-
> > >length);
> > + if (status) {
> > + pr_err("%s: failed to queue message
> > (%d)",
> > + __func__, status);
> > + }
> > +
> > + /* If no reply is needed then we're done */
> > + if (!cmd->wait) {
> > + mutex_lock(&instance->lock);
> > + list_del(&cmd->head);
> > + mutex_unlock(&instance->lock);
> > + vc_vchi_cmd_delete(instance, cmd);
> > + continue;
> > + }
> > +
> > + if (status) {
> > + complete(&cmd->cmplt);
> > + continue;
> > + }
> > +
> > + } while (1);
> > +
> > + while (!vchi_msg_peek(instance->vchi_handle[0], (void
> > **)&reply,
> > + &reply_len, VCHI_FLAGS_NONE)) {
> > + if (reply->trans_id & 0x80000000) {
> > + /* Async event or cmd from the VPU */
> > + if (instance->vpu_event)
> > + instance->vpu_event(instance,
> > reply,
> > + reply_len);
> > + } else {
> > + vc_sm_cma_vchi_rx_ack(instance, cmd,
> > reply,
> > + reply_len);
> > + }
> > +
> > + vchi_msg_remove(instance->vchi_handle[0]);
> > + }
> > +
> > + /* Go through the dead list and free them */
> > + mutex_lock(&instance->lock);
> > + list_for_each_entry_safe(cmd, cmd_tmp, &instance-
> > >dead_list,
> > + head) {
> > + list_del(&cmd->head);
> > + vc_vchi_cmd_delete(instance, cmd);
> > + }
> > + mutex_unlock(&instance->lock);
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static void vc_sm_cma_vchi_callback(void *param,
> > + const enum vchi_callback_reason
> > reason,
> > + void *msg_handle)
> > +{
> > + struct sm_instance *instance = param;
> > +
> > + (void)msg_handle;
> > +
> > + switch (reason) {
> > + case VCHI_CALLBACK_MSG_AVAILABLE:
> > + complete(&instance->io_cmplt);
> > + break;
> > +
> > + case VCHI_CALLBACK_SERVICE_CLOSED:
> > + pr_info("%s: service CLOSED!!", __func__);
> > + default:
> > + break;
> > + }
> > +}
> > +
> > +struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle
> > *vchi_instance,
> > + unsigned int num_connections,
> > + vpu_event_cb vpu_event)
> > +{
> > + u32 i;
> > + struct sm_instance *instance;
> > + int status;
> > +
> > + pr_debug("%s: start", __func__);
> > +
> > + if (num_connections > VCHI_MAX_NUM_CONNECTIONS) {
> > + pr_err("%s: unsupported number of connections %u
> > (max=%u)",
> > + __func__, num_connections,
> > VCHI_MAX_NUM_CONNECTIONS);
> > +
> > + goto err_null;
> > + }
> > + /* Allocate memory for this instance */
> > + instance = kzalloc(sizeof(*instance), GFP_KERNEL);
> > +
> > + /* Misc initialisations */
> > + mutex_init(&instance->lock);
> > + init_completion(&instance->io_cmplt);
> > + INIT_LIST_HEAD(&instance->cmd_list);
> > + INIT_LIST_HEAD(&instance->rsp_list);
> > + INIT_LIST_HEAD(&instance->dead_list);
> > + INIT_LIST_HEAD(&instance->free_list);
> > + sema_init(&instance->free_sema, SM_MAX_NUM_CMD_RSP_BLKS);
> > + mutex_init(&instance->free_lock);
> > + for (i = 0; i < SM_MAX_NUM_CMD_RSP_BLKS; i++) {
> > + init_completion(&instance->free_blk[i].cmplt);
> > + list_add(&instance->free_blk[i].head, &instance-
> > >free_list);
> > + }
> > +
> > + /* Open the VCHI service connections */
> > + instance->num_connections = num_connections;
> > + for (i = 0; i < num_connections; i++) {
> > + struct service_creation params = {
> > + .version = VCHI_VERSION_EX(VC_SM_VER,
> > VC_SM_MIN_VER),
> > + .service_id = VC_SM_SERVER_NAME,
> > + .callback = vc_sm_cma_vchi_callback,
> > + .callback_param = instance,
> > + };
> > +
> > + status = vchi_service_open(vchi_instance,
> > + ¶ms, &instance-
> > >vchi_handle[i]);
> > + if (status) {
> > + pr_err("%s: failed to open VCHI service (%d)",
> > + __func__, status);
> > +
> > + goto err_close_services;
> > + }
> > + }
> > +
> > + /* Create the thread which takes care of all io to/from
> > videoocore. */
> > + instance->io_thread =
> > kthread_create(&vc_sm_cma_vchi_videocore_io,
> > + (void *)instance, "SMIO");
> > + if (!instance->io_thread) {
> > + pr_err("%s: failed to create SMIO thread", __func__);
> > +
> > + goto err_close_services;
> > + }
> > + instance->vpu_event = vpu_event;
> > + set_user_nice(instance->io_thread, -10);
> > + wake_up_process(instance->io_thread);
> > +
> > + pr_debug("%s: success - instance %p", __func__, instance);
> > + return instance;
> > +
> > +err_close_services:
> > + for (i = 0; i < instance->num_connections; i++) {
> > + if (instance->vchi_handle[i])
> > + vchi_service_close(instance->vchi_handle[i]);
> > + }
> > + kfree(instance);
> > +err_null:
> > + pr_debug("%s: FAILED", __func__);
> > + return NULL;
> > +}
> > +
> > +int vc_sm_cma_vchi_stop(struct sm_instance **handle)
> > +{
> > + struct sm_instance *instance;
> > + u32 i;
> > +
> > + if (!handle) {
> > + pr_err("%s: invalid pointer to handle %p", __func__,
> > handle);
> > + goto lock;
> > + }
> > +
> > + if (!*handle) {
> > + pr_err("%s: invalid handle %p", __func__, *handle);
> > + goto lock;
> > + }
> > +
> > + instance = *handle;
> > +
> > + /* Close all VCHI service connections */
> > + for (i = 0; i < instance->num_connections; i++) {
> > + s32 success;
> > +
> > + vchi_service_use(instance->vchi_handle[i]);
> > +
> > + success = vchi_service_close(instance->vchi_handle[i]);
> > + }
> > +
> > + kfree(instance);
> > +
> > + *handle = NULL;
> > + return 0;
> > +
> > +lock:
> > + return -EINVAL;
> > +}
> > +
> > +static int vc_sm_cma_vchi_send_msg(struct sm_instance *handle,
> > + enum vc_sm_msg_type msg_id, void
> > *msg,
> > + u32 msg_size, void *result, u32
> > result_size,
> > + u32 *cur_trans_id, u8 wait_reply)
> > +{
> > + int status = 0;
> > + struct sm_instance *instance = handle;
> > + struct sm_cmd_rsp_blk *cmd_blk;
> > +
> > + if (!handle) {
> > + pr_err("%s: invalid handle", __func__);
> > + return -EINVAL;
> > + }
> > + if (!msg) {
> > + pr_err("%s: invalid msg pointer", __func__);
> > + return -EINVAL;
> > + }
> > +
> > + cmd_blk =
> > + vc_vchi_cmd_create(instance, msg_id, msg, msg_size,
> > wait_reply);
> > + if (!cmd_blk) {
> > + pr_err("[%s]: failed to allocate global tracking
> > resource",
> > + __func__);
> > + return -ENOMEM;
> > + }
> > +
> > + if (cur_trans_id)
> > + *cur_trans_id = cmd_blk->id;
> > +
> > + mutex_lock(&instance->lock);
> > + list_add_tail(&cmd_blk->head, &instance->cmd_list);
> > + mutex_unlock(&instance->lock);
> > + complete(&instance->io_cmplt);
> > +
> > + if (!wait_reply)
> > + /* We're done */
> > + return 0;
> > +
> > + /* Wait for the response */
> > + if (wait_for_completion_interruptible(&cmd_blk->cmplt)) {
> > + mutex_lock(&instance->lock);
> > + if (!cmd_blk->sent) {
> > + list_del(&cmd_blk->head);
> > + mutex_unlock(&instance->lock);
> > + vc_vchi_cmd_delete(instance, cmd_blk);
> > + return -ENXIO;
> > + }
> > +
> > + list_move(&cmd_blk->head, &instance->dead_list);
> > + mutex_unlock(&instance->lock);
> > + complete(&instance->io_cmplt);
> > + return -EINTR; /* We're done */
> > + }
> > +
> > + if (result && result_size) {
> > + memcpy(result, cmd_blk->msg, result_size);
> > + } else {
> > + struct vc_sm_result_t *res =
> > + (struct vc_sm_result_t *)cmd_blk->msg;
> > + status = (res->success == 0) ? 0 : -ENXIO;
> > + }
> > +
> > + mutex_lock(&instance->lock);
> > + list_del(&cmd_blk->head);
> > + mutex_unlock(&instance->lock);
> > + vc_vchi_cmd_delete(instance, cmd_blk);
> > + return status;
> > +}
> > +
> > +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct
> > vc_sm_free_t *msg,
> > + u32 *cur_trans_id)
> > +{
> > + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_FREE,
> > + msg, sizeof(*msg), 0, 0,
> > cur_trans_id, 0);
> > +}
> > +
> > +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct
> > vc_sm_import *msg,
> > + struct vc_sm_import_result *result, u32
> > *cur_trans_id)
> > +{
> > + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_IMPORT,
> > + msg, sizeof(*msg), result,
> > sizeof(*result),
> > + cur_trans_id, 1);
> > +}
> > +
> > +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> > + struct vc_sm_version *msg,
> > + struct vc_sm_result_t *result,
> > + u32 *cur_trans_id)
> > +{
> > + return vc_sm_cma_vchi_send_msg(handle,
> > VC_SM_MSG_TYPE_CLIENT_VERSION,
> > + //msg, sizeof(*msg), result,
> > sizeof(*result),
> > + //cur_trans_id, 1);
> > + msg, sizeof(*msg), NULL, 0,
> > + cur_trans_id, 0);
> > +}
> > +
> > +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> > + struct
> > vc_sm_vc_mem_request_result *msg,
> > + uint32_t *cur_trans_id)
> > +{
> > + return vc_sm_cma_vchi_send_msg(handle,
> > + VC_SM_MSG_TYPE_VC_MEM_REQUEST_RE
> > PLY,
> > + msg, sizeof(*msg), 0, 0,
> > cur_trans_id,
> > + 0);
> > +}
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> > new file mode 100644
> > index 000000000000..e8db34bd1e91
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> > @@ -0,0 +1,63 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +/*
> > + * VideoCore Shared Memory CMA allocator
> > + *
> > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> > + *
> > + * Based on vmcs_sm driver from Broadcom Corporation.
> > + *
> > + */
> > +
> > +#ifndef __VC_SM_CMA_VCHI_H__INCLUDED__
> > +#define __VC_SM_CMA_VCHI_H__INCLUDED__
> > +
> > +#include "interface/vchi/vchi.h"
> > +
> > +#include "vc_sm_defs.h"
> > +
> > +/*
> > + * Forward declare.
> > + */
> > +struct sm_instance;
> > +
> > +typedef void (*vpu_event_cb)(struct sm_instance *instance,
> > + struct vc_sm_result_t *reply, int
> > reply_len);
> > +
> > +/*
> > + * Initialize the shared memory service, opens up vchi connection to
> > talk to it.
> > + */
> > +struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle
> > *vchi_instance,
> > + unsigned int num_connections,
> > + vpu_event_cb vpu_event);
> > +
> > +/*
> > + * Terminates the shared memory service.
> > + */
> > +int vc_sm_cma_vchi_stop(struct sm_instance **handle);
> > +
> > +/*
> > + * Ask the shared memory service to free up some memory that was
> > previously
> > + * allocated by the vc_sm_cma_vchi_alloc function call.
> > + */
> > +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct
> > vc_sm_free_t *msg,
> > + u32 *cur_trans_id);
> > +
> > +/*
> > + * Import a contiguous block of memory and wrap it in a GPU
> > MEM_HANDLE_T.
> > + */
> > +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct
> > vc_sm_import *msg,
> > + struct vc_sm_import_result *result,
> > + u32 *cur_trans_id);
> > +
> > +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> > + struct vc_sm_version *msg,
> > + struct vc_sm_result_t *result,
> > + u32 *cur_trans_id);
> > +
> > +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> > + struct
> > vc_sm_vc_mem_request_result *msg,
> > + uint32_t *cur_trans_id);
> > +
> > +#endif /* __VC_SM_CMA_VCHI_H__INCLUDED__ */
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> > new file mode 100644
> > index 000000000000..8a0d1f6dbfe8
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> > @@ -0,0 +1,300 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +/*
> > + * VideoCore Shared Memory CMA allocator
> > + *
> > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > + *
> > + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom
> > Corporation.
> > + * All IPC messages are copied across to this file, even if the vc-
> > sm-cma
> > + * driver is not currently using them.
> > + *
> > +
> > *********************************************************************
> > *******
> > + */
> > +
> > +#ifndef __VC_SM_DEFS_H__INCLUDED__
> > +#define __VC_SM_DEFS_H__INCLUDED__
> > +
> > +/* FourCC code used for VCHI connection */
> > +#define VC_SM_SERVER_NAME MAKE_FOURCC("SMEM")
> > +
> > +/* Maximum message length */
> > +#define VC_SM_MAX_MSG_LEN (sizeof(union vc_sm_msg_union_t) + \
> > + sizeof(struct vc_sm_msg_hdr_t))
> > +#define VC_SM_MAX_RSP_LEN (sizeof(union vc_sm_msg_union_t))
> > +
> > +/* Resource name maximum size */
> > +#define VC_SM_RESOURCE_NAME 32
> > +
> > +/*
> > + * Version to be reported to the VPU
> > + * VPU assumes 0 (aka 1) which does not require the released
> > callback, nor
> > + * expect the client to handle VC_MEM_REQUESTS.
> > + * Version 2 requires the released callback, and must support
> > VC_MEM_REQUESTS.
> > + */
> > +#define VC_SM_PROTOCOL_VERSION 2
> > +
> > +enum vc_sm_msg_type {
> > + /* Message types supported for HOST->VC direction */
> > +
> > + /* Allocate shared memory block */
> > + VC_SM_MSG_TYPE_ALLOC,
> > + /* Lock allocated shared memory block */
> > + VC_SM_MSG_TYPE_LOCK,
> > + /* Unlock allocated shared memory block */
> > + VC_SM_MSG_TYPE_UNLOCK,
> > + /* Unlock allocated shared memory block, do not answer command
> > */
> > + VC_SM_MSG_TYPE_UNLOCK_NOANS,
> > + /* Free shared memory block */
> > + VC_SM_MSG_TYPE_FREE,
> > + /* Resize a shared memory block */
> > + VC_SM_MSG_TYPE_RESIZE,
> > + /* Walk the allocated shared memory block(s) */
> > + VC_SM_MSG_TYPE_WALK_ALLOC,
> > +
> > + /* A previously applied action will need to be reverted */
> > + VC_SM_MSG_TYPE_ACTION_CLEAN,
> > +
> > + /*
> > + * Import a physical address and wrap into a MEM_HANDLE_T.
> > + * Release with VC_SM_MSG_TYPE_FREE.
> > + */
> > + VC_SM_MSG_TYPE_IMPORT,
> > + /*
> > + *Tells VC the protocol version supported by this client.
> > + * 2 supports the async/cmd messages from the VPU for final
> > release
> > + * of memory, and for VC allocations.
> > + */
> > + VC_SM_MSG_TYPE_CLIENT_VERSION,
> > + /* Response to VC request for memory */
> > + VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
> > +
> > + /*
> > + * Asynchronous/cmd messages supported for VC->HOST direction.
> > + * Signalled by setting the top bit in vc_sm_result_t trans_id.
> > + */
> > +
> > + /*
> > + * VC has finished with an imported memory allocation.
> > + * Release any Linux reference counts on the underlying block.
> > + */
> > + VC_SM_MSG_TYPE_RELEASED,
> > + /* VC request for memory */
> > + VC_SM_MSG_TYPE_VC_MEM_REQUEST,
> > +
> > + VC_SM_MSG_TYPE_MAX
> > +};
> > +
> > +/* Type of memory to be allocated */
> > +enum vc_sm_alloc_type_t {
> > + VC_SM_ALLOC_CACHED,
> > + VC_SM_ALLOC_NON_CACHED,
> > +};
> > +
> > +/* Message header for all messages in HOST->VC direction */
> > +struct vc_sm_msg_hdr_t {
> > + u32 type;
> > + u32 trans_id;
> > + u8 body[0];
> > +
> > +};
> > +
> > +/* Request to allocate memory (HOST->VC) */
> > +struct vc_sm_alloc_t {
> > + /* type of memory to allocate */
> > + enum vc_sm_alloc_type_t type;
> > + /* byte amount of data to allocate per unit */
> > + u32 base_unit;
> > + /* number of unit to allocate */
> > + u32 num_unit;
> > + /* alignment to be applied on allocation */
> > + u32 alignment;
> > + /* identity of who allocated this block */
> > + u32 allocator;
> > + /* resource name (for easier tracking on vc side) */
> > + char name[VC_SM_RESOURCE_NAME];
> > +
> > +};
> > +
> > +/* Result of a requested memory allocation (VC->HOST) */
> > +struct vc_sm_alloc_result_t {
> > + /* Transaction identifier */
> > + u32 trans_id;
> > +
> > + /* Resource handle */
> > + u32 res_handle;
> > + /* Pointer to resource buffer */
> > + u32 res_mem;
> > + /* Resource base size (bytes) */
> > + u32 res_base_size;
> > + /* Resource number */
> > + u32 res_num;
> > +
> > +};
> > +
> > +/* Request to free a previously allocated memory (HOST->VC) */
> > +struct vc_sm_free_t {
> > + /* Resource handle (returned from alloc) */
> > + u32 res_handle;
> > + /* Resource buffer (returned from alloc) */
> > + u32 res_mem;
> > +
> > +};
> > +
> > +/* Request to lock a previously allocated memory (HOST->VC) */
> > +struct vc_sm_lock_unlock_t {
> > + /* Resource handle (returned from alloc) */
> > + u32 res_handle;
> > + /* Resource buffer (returned from alloc) */
> > + u32 res_mem;
> > +
> > +};
> > +
> > +/* Request to resize a previously allocated memory (HOST->VC) */
> > +struct vc_sm_resize_t {
> > + /* Resource handle (returned from alloc) */
> > + u32 res_handle;
> > + /* Resource buffer (returned from alloc) */
> > + u32 res_mem;
> > + /* Resource *new* size requested (bytes) */
> > + u32 res_new_size;
> > +
> > +};
> > +
> > +/* Result of a requested memory lock (VC->HOST) */
> > +struct vc_sm_lock_result_t {
> > + /* Transaction identifier */
> > + u32 trans_id;
> > +
> > + /* Resource handle */
> > + u32 res_handle;
> > + /* Pointer to resource buffer */
> > + u32 res_mem;
> > + /*
> > + * Pointer to former resource buffer if the memory
> > + * was reallocated
> > + */
> > + u32 res_old_mem;
> > +
> > +};
> > +
> > +/* Generic result for a request (VC->HOST) */
> > +struct vc_sm_result_t {
> > + /* Transaction identifier */
> > + u32 trans_id;
> > +
> > + s32 success;
> > +
> > +};
> > +
> > +/* Request to revert a previously applied action (HOST->VC) */
> > +struct vc_sm_action_clean_t {
> > + /* Action of interest */
> > + enum vc_sm_msg_type res_action;
> > + /* Transaction identifier for the action of interest */
> > + u32 action_trans_id;
> > +
> > +};
> > +
> > +/* Request to remove all data associated with a given allocator
> > (HOST->VC) */
> > +struct vc_sm_free_all_t {
> > + /* Allocator identifier */
> > + u32 allocator;
> > +};
> > +
> > +/* Request to import memory (HOST->VC) */
> > +struct vc_sm_import {
> > + /* type of memory to allocate */
> > + enum vc_sm_alloc_type_t type;
> > + /* pointer to the VC (ie physical) address of the allocated
> > memory */
> > + u32 addr;
> > + /* size of buffer */
> > + u32 size;
> > + /* opaque handle returned in RELEASED messages */
> > + u32 kernel_id;
> > + /* Allocator identifier */
> > + u32 allocator;
> > + /* resource name (for easier tracking on vc side) */
> > + char name[VC_SM_RESOURCE_NAME];
> > +};
> > +
> > +/* Result of a requested memory import (VC->HOST) */
> > +struct vc_sm_import_result {
> > + /* Transaction identifier */
> > + u32 trans_id;
> > +
> > + /* Resource handle */
> > + u32 res_handle;
> > +};
> > +
> > +/* Notification that VC has finished with an allocation (VC->HOST)
> > */
> > +struct vc_sm_released {
> > + /* cmd type / trans_id */
> > + u32 cmd;
> > +
> > + /* pointer to the VC (ie physical) address of the allocated
> > memory */
> > + u32 addr;
> > + /* size of buffer */
> > + u32 size;
> > + /* opaque handle returned in RELEASED messages */
> > + u32 kernel_id;
> > + u32 vc_handle;
> > +};
> > +
> > +/*
> > + * Client informing VC as to the protocol version it supports.
> > + * >=2 requires the released callback, and supports VC asking for
> > memory.
> > + * Failure means that the firmware doesn't support this call, and
> > therefore the
> > + * client should either fail, or NOT rely on getting the released
> > callback.
> > + */
> > +struct vc_sm_version {
> > + u32 version;
> > +};
> > +
> > +/* Request FROM VideoCore for some memory */
> > +struct vc_sm_vc_mem_request {
> > + /* cmd type */
> > + u32 cmd;
> > +
> > + /* trans_id (from VPU) */
> > + u32 trans_id;
> > + /* size of buffer */
> > + u32 size;
> > + /* alignment of buffer */
> > + u32 align;
> > + /* resource name (for easier tracking) */
> > + char name[VC_SM_RESOURCE_NAME];
> > + /* VPU handle for the resource */
> > + u32 vc_handle;
> > +};
> > +
> > +/* Response from the kernel to provide the VPU with some memory */
> > +struct vc_sm_vc_mem_request_result {
> > + /* Transaction identifier for the VPU */
> > + u32 trans_id;
> > + /* pointer to the physical address of the allocated memory */
> > + u32 addr;
> > + /* opaque handle returned in RELEASED messages */
> > + u32 kernel_id;
> > +};
> > +
> > +/* Union of ALL messages */
> > +union vc_sm_msg_union_t {
> > + struct vc_sm_alloc_t alloc;
> > + struct vc_sm_alloc_result_t alloc_result;
> > + struct vc_sm_free_t free;
> > + struct vc_sm_lock_unlock_t lock_unlock;
> > + struct vc_sm_action_clean_t action_clean;
> > + struct vc_sm_resize_t resize;
> > + struct vc_sm_lock_result_t lock_result;
> > + struct vc_sm_result_t result;
> > + struct vc_sm_free_all_t free_all;
> > + struct vc_sm_import import;
> > + struct vc_sm_import_result import_result;
> > + struct vc_sm_version version;
> > + struct vc_sm_released released;
> > + struct vc_sm_vc_mem_request vc_request;
> > + struct vc_sm_vc_mem_request_result vc_request_result;
> > +};
> > +
> > +#endif /* __VC_SM_DEFS_H__INCLUDED__ */
> > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> > new file mode 100644
> > index 000000000000..988fdd967922
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> > @@ -0,0 +1,28 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +
> > +/*
> > + * VideoCore Shared Memory CMA allocator
> > + *
> > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > + *
> > + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom
> > Corporation.
> > + *
> > + */
> > +
> > +#ifndef __VC_SM_KNL_H__INCLUDED__
> > +#define __VC_SM_KNL_H__INCLUDED__
> > +
> > +#if !defined(__KERNEL__)
> > +#error "This interface is for kernel use only..."
> > +#endif
> > +
> > +/* Free a previously allocated or imported shared memory handle and
> > block. */
> > +int vc_sm_cma_free(void *handle);
> > +
> > +/* Get an internal resource handle mapped from the external one. */
> > +int vc_sm_cma_int_handle(void *handle);
> > +
> > +/* Import a block of memory into the GPU space. */
> > +int vc_sm_cma_import_dmabuf(struct dma_buf *dmabuf, void **handle);
> > +
> > +#endif /* __VC_SM_KNL_H__INCLUDED__ */
>
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-06 19:24 ` Dave Stevenson
@ 2020-05-08 0:11 ` Laurent Pinchart
2020-05-18 12:06 ` Hans Verkuil
2020-08-24 16:39 ` Jacopo Mondi
1 sibling, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-08 0:11 UTC (permalink / raw)
To: Dave Stevenson
Cc: Nicolas Saenz Julienne, Linux Media Mailing List, Dave Stevenson,
Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck
Hi Dave,
On Wed, May 06, 2020 at 08:24:38PM +0100, Dave Stevenson wrote:
> On Wed, 6 May 2020 at 19:04, Nicolas Saenz Julienne wrote:
> > On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> >> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> >>
> >> Add Broadcom VideoCore Shared Memory support.
> >>
> >> This new driver allows contiguous memory blocks to be imported
> >> into the VideoCore VPU memory map, and manages the lifetime of
> >> those objects, only releasing the source dmabuf once the VPU has
> >> confirmed it has finished with it.
> >
> > I'm still digesting all this, but a question came up, who is using the
> > ioctls?
>
> We have a userspace library that uses it [1].
> It is used by things like MMAL to share buffers between the VPU and
> ARM, rather than having to get VCHI to copy all the data between
> mirrored buffers.
>
> I think what has happened here is that Laurent has picked up the
> version of the driver from the top of our downstream kernel tree.
> For libcamera and the ISP driver, we need a significantly smaller
> feature set, basically import of dmabufs only, no allocations or cache
> management. For the ISP driver it's mainly dmabuf import from
> videobuf2 for the image buffers, but there's also a need to pass in
> lens shading tables which are relatively large. With a small amount of
> rework in libcamera, we can make it so that we use dma-buf heaps to do
> the allocation, and pass in a dmabuf fd to the ISP driver to then map
> onto the VPU. That removes all the ioctls handling from this driver.
>
> Downstream we do have other use cases that want to be able to do other
> functions on shared memory, but that too should be reworkable into
> using dma-buf heaps for allocations, and vcsm only handles importing
> dmabufs via an ioctl. All that can be hidden away in the vcsm library,
> so applications don't care.
> We've also got some legacy code kicking around, as there was
> originally a version of the driver that mapped the VPU's memory blocks
> to the ARM. That's why the vcsm library has two code paths through
> almost every function - one for each driver.
>
> Laurent: What's your view? Halt the review this particular patch for
> now and rework, or try and get this all integrated?
> Mainline obviously already has dma-buf heaps merged, whilst I have a
> PR cherry-picking it back into our downstream 5.4. The main reason it
> hasn't been merged is that I haven't had a test case to prove it
> works. The rework should be relatively simple, but will need small
> updates to both libcamera and ISP driver.
My preference would be to go for a solution based on dma-buf heap right
away for mainline, to minimize the code going into staging. There's no
hurry there, and I can help integrating the changes in libcamera if
needed.
> [1] https://github.com/raspberrypi/userland/tree/master/host_applications/linux/libs/sm
>
> >> Driver upported from the RaspberryPi BSP at revision:
> >> 890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when
> >> counting transactions")
> >> forward ported to recent mainline kernel version.
> >>
> >> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> >> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> >> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> >> ---
> >> drivers/staging/vc04_services/Kconfig | 2 +
> >> drivers/staging/vc04_services/Makefile | 1 +
> >> .../include/linux/broadcom/vc_sm_cma_ioctl.h | 114 ++
> >> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> >> .../staging/vc04_services/vc-sm-cma/Makefile | 13 +
> >> drivers/staging/vc04_services/vc-sm-cma/TODO | 1 +
> >> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 1732 +++++++++++++++++
> >> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 84 +
> >> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 505 +++++
> >> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
> >> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 300 +++
> >> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> >> 12 files changed, 2853 insertions(+)
> >> create mode 100644
> >> drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.h
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/TODO
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> >> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-08 0:11 ` Laurent Pinchart
@ 2020-05-18 12:06 ` Hans Verkuil
0 siblings, 0 replies; 111+ messages in thread
From: Hans Verkuil @ 2020-05-18 12:06 UTC (permalink / raw)
To: Laurent Pinchart, Dave Stevenson
Cc: Nicolas Saenz Julienne, Linux Media Mailing List, Dave Stevenson,
Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck
On 08/05/2020 02:11, Laurent Pinchart wrote:
> Hi Dave,
>
> On Wed, May 06, 2020 at 08:24:38PM +0100, Dave Stevenson wrote:
>> On Wed, 6 May 2020 at 19:04, Nicolas Saenz Julienne wrote:
>>> On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
>>>> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>>>>
>>>> Add Broadcom VideoCore Shared Memory support.
>>>>
>>>> This new driver allows contiguous memory blocks to be imported
>>>> into the VideoCore VPU memory map, and manages the lifetime of
>>>> those objects, only releasing the source dmabuf once the VPU has
>>>> confirmed it has finished with it.
>>>
>>> I'm still digesting all this, but a question came up, who is using the
>>> ioctls?
>>
>> We have a userspace library that uses it [1].
>> It is used by things like MMAL to share buffers between the VPU and
>> ARM, rather than having to get VCHI to copy all the data between
>> mirrored buffers.
>>
>> I think what has happened here is that Laurent has picked up the
>> version of the driver from the top of our downstream kernel tree.
>> For libcamera and the ISP driver, we need a significantly smaller
>> feature set, basically import of dmabufs only, no allocations or cache
>> management. For the ISP driver it's mainly dmabuf import from
>> videobuf2 for the image buffers, but there's also a need to pass in
>> lens shading tables which are relatively large. With a small amount of
>> rework in libcamera, we can make it so that we use dma-buf heaps to do
>> the allocation, and pass in a dmabuf fd to the ISP driver to then map
>> onto the VPU. That removes all the ioctls handling from this driver.
>>
>> Downstream we do have other use cases that want to be able to do other
>> functions on shared memory, but that too should be reworkable into
>> using dma-buf heaps for allocations, and vcsm only handles importing
>> dmabufs via an ioctl. All that can be hidden away in the vcsm library,
>> so applications don't care.
>> We've also got some legacy code kicking around, as there was
>> originally a version of the driver that mapped the VPU's memory blocks
>> to the ARM. That's why the vcsm library has two code paths through
>> almost every function - one for each driver.
>>
>> Laurent: What's your view? Halt the review this particular patch for
>> now and rework, or try and get this all integrated?
>> Mainline obviously already has dma-buf heaps merged, whilst I have a
>> PR cherry-picking it back into our downstream 5.4. The main reason it
>> hasn't been merged is that I haven't had a test case to prove it
>> works. The rework should be relatively simple, but will need small
>> updates to both libcamera and ISP driver.
>
> My preference would be to go for a solution based on dma-buf heap right
> away for mainline, to minimize the code going into staging. There's no
> hurry there, and I can help integrating the changes in libcamera if
> needed.
So just to be clear, these vc-sm-cma ioctls will disappear in the next version?
I had the same question while reviewing this as Nicolas had :-)
Regards,
Hans
>
>> [1] https://github.com/raspberrypi/userland/tree/master/host_applications/linux/libs/sm
>>
>>>> Driver upported from the RaspberryPi BSP at revision:
>>>> 890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when
>>>> counting transactions")
>>>> forward ported to recent mainline kernel version.
>>>>
>>>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
>>>> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
>>>> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
>>>> ---
>>>> drivers/staging/vc04_services/Kconfig | 2 +
>>>> drivers/staging/vc04_services/Makefile | 1 +
>>>> .../include/linux/broadcom/vc_sm_cma_ioctl.h | 114 ++
>>>> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
>>>> .../staging/vc04_services/vc-sm-cma/Makefile | 13 +
>>>> drivers/staging/vc04_services/vc-sm-cma/TODO | 1 +
>>>> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 1732 +++++++++++++++++
>>>> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 84 +
>>>> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 505 +++++
>>>> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
>>>> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 300 +++
>>>> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
>>>> 12 files changed, 2853 insertions(+)
>>>> create mode 100644
>>>> drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.h
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/TODO
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
>>>> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
>
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-06 19:24 ` Dave Stevenson
2020-05-08 0:11 ` Laurent Pinchart
@ 2020-08-24 16:39 ` Jacopo Mondi
2020-08-25 17:52 ` Dave Stevenson
1 sibling, 1 reply; 111+ messages in thread
From: Jacopo Mondi @ 2020-08-24 16:39 UTC (permalink / raw)
To: Dave Stevenson
Cc: Nicolas Saenz Julienne, Laurent Pinchart,
Linux Media Mailing List, Dave Stevenson, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
Hi Dave, Nicolas, Laurent,
On Wed, May 06, 2020 at 08:24:38PM +0100, Dave Stevenson wrote:
> Hi Nicolas
>
> On Wed, 6 May 2020 at 19:04, Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
> >
> > Hi Laurent, Dave,
> >
> > On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > >
> > > Add Broadcom VideoCore Shared Memory support.
> > >
> > > This new driver allows contiguous memory blocks to be imported
> > > into the VideoCore VPU memory map, and manages the lifetime of
> > > those objects, only releasing the source dmabuf once the VPU has
> > > confirmed it has finished with it.
> > >
> >
> > I'm still digesting all this, but a question came up, who is using the
> > ioctls?
>
> We have a userspace library that uses it [1].
> It is used by things like MMAL to share buffers between the VPU and
> ARM, rather than having to get VCHI to copy all the data between
> mirrored buffers.
>
> I think what has happened here is that Laurent has picked up the
> version of the driver from the top of our downstream kernel tree.
> For libcamera and the ISP driver, we need a significantly smaller
> feature set, basically import of dmabufs only, no allocations or cache
> management. For the ISP driver it's mainly dmabuf import from
> videobuf2 for the image buffers, but there's also a need to pass in
> lens shading tables which are relatively large. With a small amount of
> rework in libcamera, we can make it so that we use dma-buf heaps to do
> the allocation, and pass in a dmabuf fd to the ISP driver to then map
> onto the VPU. That removes all the ioctls handling from this driver.
>
> Downstream we do have other use cases that want to be able to do other
> functions on shared memory, but that too should be reworkable into
> using dma-buf heaps for allocations, and vcsm only handles importing
> dmabufs via an ioctl. All that can be hidden away in the vcsm library,
> so applications don't care.
> We've also got some legacy code kicking around, as there was
> originally a version of the driver that mapped the VPU's memory blocks
> to the ARM. That's why the vcsm library has two code paths through
> almost every function - one for each driver.
>
> Laurent: What's your view? Halt the review this particular patch for
> now and rework, or try and get this all integrated?
> Mainline obviously already has dma-buf heaps merged, whilst I have a
> PR cherry-picking it back into our downstream 5.4. The main reason it
> hasn't been merged is that I haven't had a test case to prove it
> works. The rework should be relatively simple, but will need small
> updates to both libcamera and ISP driver.
As months have passed, libcamera moved to allocate lens shading tables
using dma-buf heaps and the only user I can name of the vc-sm-cma
driver is the actual ISP, that needs to import the dmabuf pointing to
the lens shading maps with vc_sm_cma_import_dmabuf().
Upstreaming the whole vc-sm-cma driver as it is for this single kAPI
seems a bit a no-go. Dave, what would you prefer here ? Should I
provide a minimal vc-sm-cam driver that only performs buffer importing
to support the ISP driver ? Is the buffer importing into VPU there to
stay or is its usage transitional and can be kept out of the next
submission of this series ?
Thanks
j
>
> Dave
>
> [1] https://github.com/raspberrypi/userland/tree/master/host_applications/linux/libs/sm
>
> > Regards,
> > Nicolas
> >
> > > Driver upported from the RaspberryPi BSP at revision:
> > > 890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when
> > > counting transactions")
> > > forward ported to recent mainline kernel version.
> > >
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> > > drivers/staging/vc04_services/Kconfig | 2 +
> > > drivers/staging/vc04_services/Makefile | 1 +
> > > .../include/linux/broadcom/vc_sm_cma_ioctl.h | 114 ++
> > > .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> > > .../staging/vc04_services/vc-sm-cma/Makefile | 13 +
> > > drivers/staging/vc04_services/vc-sm-cma/TODO | 1 +
> > > .../staging/vc04_services/vc-sm-cma/vc_sm.c | 1732
> > > +++++++++++++++++
> > > .../staging/vc04_services/vc-sm-cma/vc_sm.h | 84 +
> > > .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 505 +++++
> > > .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
> > > .../vc04_services/vc-sm-cma/vc_sm_defs.h | 300 +++
> > > .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> > > 12 files changed, 2853 insertions(+)
> > > create mode 100644
> > > drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.
> > > h
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/TODO
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > > cma/vc_sm_cma_vchi.c
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > > cma/vc_sm_cma_vchi.h
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > > cma/vc_sm_defs.h
> > > create mode 100644 drivers/staging/vc04_services/vc-sm-
> > > cma/vc_sm_knl.h
> > >
> > > diff --git a/drivers/staging/vc04_services/Kconfig
> > > b/drivers/staging/vc04_services/Kconfig
> > > index 6baf9dd57f1f..b9f1f019ebd8 100644
> > > --- a/drivers/staging/vc04_services/Kconfig
> > > +++ b/drivers/staging/vc04_services/Kconfig
> > > @@ -23,5 +23,7 @@ source "drivers/staging/vc04_services/bcm2835-
> > > audio/Kconfig"
> > >
> > > source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
> > >
> > > +source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
> > > +
> > > endif
> > >
> > > diff --git a/drivers/staging/vc04_services/Makefile
> > > b/drivers/staging/vc04_services/Makefile
> > > index 54d9e2f31916..6e1abf494c1a 100644
> > > --- a/drivers/staging/vc04_services/Makefile
> > > +++ b/drivers/staging/vc04_services/Makefile
> > > @@ -12,6 +12,7 @@ vchiq-objs := \
> > >
> > > obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
> > > obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
> > > +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
> > >
> > > ccflags-y += -D__VCCOREVER__=0x04000000
> > >
> > > diff --git
> > > a/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> > > l.h
> > > b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> > > l.h
> > > new file mode 100644
> > > index 000000000000..107460ad1be3
> > > --- /dev/null
> > > +++
> > > b/drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioct
> > > l.h
> > > @@ -0,0 +1,114 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +/*
> > > + * Copyright 2019 Raspberry Pi (Trading) Ltd. All rights reserved.
> > > + *
> > > + * Based on vmcs_sm_ioctl.h Copyright Broadcom Corporation.
> > > + */
> > > +
> > > +#ifndef __VC_SM_CMA_IOCTL_H
> > > +#define __VC_SM_CMA_IOCTL_H
> > > +
> > > +/* ---- Include Files ----------------------------------------------
> > > ------ */
> > > +
> > > +#if defined(__KERNEL__)
> > > +#include <linux/types.h> /* Needed for standard types */
> > > +#else
> > > +#include <stdint.h>
> > > +#endif
> > > +
> > > +#include <linux/ioctl.h>
> > > +
> > > +/* ---- Constants and Types ----------------------------------------
> > > ------ */
> > > +
> > > +#define VC_SM_CMA_RESOURCE_NAME 32
> > > +#define VC_SM_CMA_RESOURCE_NAME_DEFAULT "sm-host-resource"
> > > +
> > > +/* Type define used to create unique IOCTL number */
> > > +#define VC_SM_CMA_MAGIC_TYPE 'J'
> > > +
> > > +/* IOCTL commands on /dev/vc-sm-cma */
> > > +enum vc_sm_cma_cmd_e {
> > > + VC_SM_CMA_CMD_ALLOC = 0x5A, /* Start at 0x5A arbitrarily
> > > */
> > > +
> > > + VC_SM_CMA_CMD_IMPORT_DMABUF,
> > > +
> > > + VC_SM_CMA_CMD_CLEAN_INVALID2,
> > > +
> > > + VC_SM_CMA_CMD_LAST /* Do not delete */
> > > +};
> > > +
> > > +/* Cache type supported, conveniently matches the user space
> > > definition in
> > > + * user-vcsm.h.
> > > + */
> > > +enum vc_sm_cma_cache_e {
> > > + VC_SM_CMA_CACHE_NONE,
> > > + VC_SM_CMA_CACHE_HOST,
> > > + VC_SM_CMA_CACHE_VC,
> > > + VC_SM_CMA_CACHE_BOTH,
> > > +};
> > > +
> > > +/* IOCTL Data structures */
> > > +struct vc_sm_cma_ioctl_alloc {
> > > + /* user -> kernel */
> > > + __u32 size;
> > > + __u32 num;
> > > + __u32 cached; /* enum vc_sm_cma_cache_e */
> > > + __u32 pad;
> > > + __u8 name[VC_SM_CMA_RESOURCE_NAME];
> > > +
> > > + /* kernel -> user */
> > > + __s32 handle;
> > > + __u32 vc_handle;
> > > + __u64 dma_addr;
> > > +};
> > > +
> > > +struct vc_sm_cma_ioctl_import_dmabuf {
> > > + /* user -> kernel */
> > > + __s32 dmabuf_fd;
> > > + __u32 cached; /* enum vc_sm_cma_cache_e */
> > > + __u8 name[VC_SM_CMA_RESOURCE_NAME];
> > > +
> > > + /* kernel -> user */
> > > + __s32 handle;
> > > + __u32 vc_handle;
> > > + __u32 size;
> > > + __u32 pad;
> > > + __u64 dma_addr;
> > > +};
> > > +
> > > +/*
> > > + * Cache functions to be set to struct
> > > vc_sm_cma_ioctl_clean_invalid2
> > > + * invalidate_mode.
> > > + */
> > > +#define VC_SM_CACHE_OP_NOP 0x00
> > > +#define VC_SM_CACHE_OP_INV 0x01
> > > +#define VC_SM_CACHE_OP_CLEAN 0x02
> > > +#define VC_SM_CACHE_OP_FLUSH 0x03
> > > +
> > > +struct vc_sm_cma_ioctl_clean_invalid2 {
> > > + __u32 op_count;
> > > + __u32 pad;
> > > + struct vc_sm_cma_ioctl_clean_invalid_block {
> > > + __u32 invalidate_mode;
> > > + __u32 block_count;
> > > + void * __user start_address;
> > > + __u32 block_size;
> > > + __u32 inter_block_stride;
> > > + } s[0];
> > > +};
> > > +
> > > +/* IOCTL numbers */
> > > +#define VC_SM_CMA_IOCTL_MEM_ALLOC\
> > > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_ALLOC,\
> > > + struct vc_sm_cma_ioctl_alloc)
> > > +
> > > +#define VC_SM_CMA_IOCTL_MEM_IMPORT_DMABUF\
> > > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_IMPORT_DMABUF,\
> > > + struct vc_sm_cma_ioctl_import_dmabuf)
> > > +
> > > +#define VC_SM_CMA_IOCTL_MEM_CLEAN_INVALID2\
> > > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
> > > + struct vc_sm_cma_ioctl_clean_invalid2)
> > > +
> > > +#endif /* __VC_SM_CMA_IOCTL_H */
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > > b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > > new file mode 100644
> > > index 000000000000..5ac115da6b49
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/Kconfig
> > > @@ -0,0 +1,10 @@
> > > +config BCM_VC_SM_CMA
> > > + bool "VideoCore Shared Memory (CMA) driver"
> > > + select BCM2835_VCHIQ
> > > + select RBTREE
> > > + select DMA_SHARED_BUFFER
> > > + help
> > > + Say Y here to enable the shared memory interface that
> > > + supports sharing dmabufs with VideoCore.
> > > + This operates over the VCHIQ interface to a service
> > > + running on VideoCore.
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/Makefile
> > > b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> > > new file mode 100644
> > > index 000000000000..77d173694fbf
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/Makefile
> > > @@ -0,0 +1,13 @@
> > > +ccflags-y += \
> > > + -I$(srctree)/$(src)/../ \
> > > + -I$(srctree)/$(src)/../interface/vchi \
> > > + -I$(srctree)/$(src)/../interface/vchiq_arm\
> > > + -I$(srctree)/$(src)/../include
> > > +
> > > +ccflags-y += \
> > > + -D__VCCOREVER__=0
> > > +
> > > +vc-sm-cma-$(CONFIG_BCM_VC_SM_CMA) := \
> > > + vc_sm.o vc_sm_cma_vchi.o
> > > +
> > > +obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma.o
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/TODO
> > > b/drivers/staging/vc04_services/vc-sm-cma/TODO
> > > new file mode 100644
> > > index 000000000000..ac9b5f8a7389
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/TODO
> > > @@ -0,0 +1 @@
> > > +No currently outstanding tasks except some clean-up.
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > > new file mode 100644
> > > index 000000000000..cd5fb561debb
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> > > @@ -0,0 +1,1732 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * VideoCore Shared Memory driver using CMA.
> > > + *
> > > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > > + * Dave Stevenson <dave.stevenson@raspberrypi.org>
> > > + *
> > > + * Based on vmcs_sm driver from Broadcom Corporation for some API,
> > > + * and taking some code for buffer allocation and dmabuf handling
> > > from
> > > + * videobuf2.
> > > + *
> > > + *
> > > + * This driver has 3 main uses:
> > > + * 1) Allocating buffers for the kernel or userspace that can be
> > > shared with the
> > > + * VPU.
> > > + * 2) Importing dmabufs from elsewhere for sharing with the VPU.
> > > + * 3) Allocating buffers for use by the VPU.
> > > + *
> > > + * In the first and second cases the native handle is a dmabuf.
> > > Releasing the
> > > + * resource inherently comes from releasing the dmabuf, and this
> > > will trigger
> > > + * unmapping on the VPU. The underlying allocation and our buffer
> > > structure are
> > > + * retained until the VPU has confirmed that it has finished with
> > > it.
> > > + *
> > > + * For the VPU allocations the VPU is responsible for triggering the
> > > release,
> > > + * and therefore the released message decrements the dma_buf
> > > refcount (with the
> > > + * VPU mapping having already been marked as released).
> > > + */
> > > +
> > > +/* ---- Include Files ----------------------------------------------
> > > ------- */
> > > +#include <linux/cdev.h>
> > > +#include <linux/device.h>
> > > +#include <linux/debugfs.h>
> > > +#include <linux/dma-mapping.h>
> > > +#include <linux/dma-buf.h>
> > > +#include <linux/errno.h>
> > > +#include <linux/fs.h>
> > > +#include <linux/kernel.h>
> > > +#include <linux/list.h>
> > > +#include <linux/miscdevice.h>
> > > +#include <linux/module.h>
> > > +#include <linux/mm.h>
> > > +#include <linux/of_device.h>
> > > +#include <linux/platform_device.h>
> > > +#include <linux/proc_fs.h>
> > > +#include <linux/slab.h>
> > > +#include <linux/seq_file.h>
> > > +#include <linux/syscalls.h>
> > > +#include <linux/types.h>
> > > +#include <asm/cacheflush.h>
> > > +
> > > +#include "vchiq_connected.h"
> > > +#include "vc_sm_cma_vchi.h"
> > > +
> > > +#include "vc_sm.h"
> > > +#include "vc_sm_knl.h"
> > > +#include <linux/broadcom/vc_sm_cma_ioctl.h>
> > > +
> > > +/* ---- Private Constants and Types --------------------------------
> > > ------- */
> > > +
> > > +#define DEVICE_NAME "vcsm-cma"
> > > +#define DEVICE_MINOR 0
> > > +
> > > +#define VC_SM_RESOURCE_NAME_DEFAULT "sm-host-resource"
> > > +
> > > +#define VC_SM_DIR_ROOT_NAME "vcsm-cma"
> > > +#define VC_SM_STATE "state"
> > > +
> > > +/* Private file data associated with each opened device. */
> > > +struct vc_sm_privdata_t {
> > > + pid_t pid; /* PID of creator. */
> > > +
> > > + int restart_sys; /* Tracks restart on interrupt. */
> > > + enum vc_sm_msg_type int_action; /* Interrupted action. */
> > > + u32 int_trans_id; /* Interrupted transaction. */
> > > +};
> > > +
> > > +typedef int (*VC_SM_SHOW) (struct seq_file *s, void *v);
> > > +struct sm_pde_t {
> > > + VC_SM_SHOW show; /* Debug fs function hookup. */
> > > + struct dentry *dir_entry; /* Debug fs directory entry. */
> > > + void *priv_data; /* Private data */
> > > +};
> > > +
> > > +/* Global state information. */
> > > +struct sm_state_t {
> > > + struct platform_device *pdev;
> > > +
> > > + struct miscdevice misc_dev;
> > > +
> > > + struct sm_instance *sm_handle; /* Handle for videocore
> > > service. */
> > > +
> > > + spinlock_t kernelid_map_lock; /* Spinlock protecting
> > > kernelid_map */
> > > + struct idr kernelid_map;
> > > +
> > > + struct mutex map_lock; /* Global map lock. */
> > > + struct list_head buffer_list; /* List of buffer. */
> > > +
> > > + struct vc_sm_privdata_t *data_knl; /* Kernel internal data
> > > tracking. */
> > > + struct vc_sm_privdata_t *vpu_allocs; /* All allocations from
> > > the VPU */
> > > + struct dentry *dir_root; /* Debug fs entries root. */
> > > + struct sm_pde_t dir_state; /* Debug fs entries state
> > > sub-tree. */
> > > +
> > > + bool require_released_callback; /* VPU will send a released
> > > msg when it
> > > + * has finished with a
> > > resource.
> > > + */
> > > + u32 int_trans_id; /* Interrupted transaction. */
> > > +};
> > > +
> > > +struct vc_sm_dma_buf_attachment {
> > > + struct device *dev;
> > > + struct sg_table sg_table;
> > > + struct list_head list;
> > > + enum dma_data_direction dma_dir;
> > > +};
> > > +
> > > +/* ---- Private Variables ------------------------------------------
> > > ----- */
> > > +
> > > +static struct sm_state_t *sm_state;
> > > +static int sm_inited;
> > > +
> > > +/* ---- Private Function Prototypes --------------------------------
> > > ------ */
> > > +
> > > +/* ---- Private Functions ------------------------------------------
> > > ------ */
> > > +
> > > +static int get_kernel_id(struct vc_sm_buffer *buffer)
> > > +{
> > > + int handle;
> > > +
> > > + spin_lock(&sm_state->kernelid_map_lock);
> > > + handle = idr_alloc(&sm_state->kernelid_map, buffer, 0, 0,
> > > GFP_KERNEL);
> > > + spin_unlock(&sm_state->kernelid_map_lock);
> > > +
> > > + return handle;
> > > +}
> > > +
> > > +static struct vc_sm_buffer *lookup_kernel_id(int handle)
> > > +{
> > > + return idr_find(&sm_state->kernelid_map, handle);
> > > +}
> > > +
> > > +static void free_kernel_id(int handle)
> > > +{
> > > + spin_lock(&sm_state->kernelid_map_lock);
> > > + idr_remove(&sm_state->kernelid_map, handle);
> > > + spin_unlock(&sm_state->kernelid_map_lock);
> > > +}
> > > +
> > > +static int vc_sm_cma_seq_file_show(struct seq_file *s, void *v)
> > > +{
> > > + struct sm_pde_t *sm_pde;
> > > +
> > > + sm_pde = (struct sm_pde_t *)(s->private);
> > > +
> > > + if (sm_pde && sm_pde->show)
> > > + sm_pde->show(s, v);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int vc_sm_cma_single_open(struct inode *inode, struct file
> > > *file)
> > > +{
> > > + return single_open(file, vc_sm_cma_seq_file_show, inode-
> > > >i_private);
> > > +}
> > > +
> > > +static const struct file_operations vc_sm_cma_debug_fs_fops = {
> > > + .open = vc_sm_cma_single_open,
> > > + .read = seq_read,
> > > + .llseek = seq_lseek,
> > > + .release = single_release,
> > > +};
> > > +
> > > +static int vc_sm_cma_global_state_show(struct seq_file *s, void *v)
> > > +{
> > > + struct vc_sm_buffer *resource = NULL;
> > > + int resource_count = 0;
> > > +
> > > + if (!sm_state)
> > > + return 0;
> > > +
> > > + seq_printf(s, "\nVC-ServiceHandle %p\n", sm_state-
> > > >sm_handle);
> > > +
> > > + /* Log all applicable mapping(s). */
> > > +
> > > + mutex_lock(&sm_state->map_lock);
> > > + seq_puts(s, "\nResources\n");
> > > + if (!list_empty(&sm_state->buffer_list)) {
> > > + list_for_each_entry(resource, &sm_state->buffer_list,
> > > + global_buffer_list) {
> > > + resource_count++;
> > > +
> > > + seq_printf(s, "\nResource %p\n",
> > > + resource);
> > > + seq_printf(s, " NAME %s\n",
> > > + resource->name);
> > > + seq_printf(s, " SIZE %zu\n",
> > > + resource->size);
> > > + seq_printf(s, " DMABUF %p\n",
> > > + resource->dma_buf);
> > > + if (resource->imported) {
> > > + seq_printf(s,
> > > " ATTACH %p\n",
> > > + resource->import.attach);
> > > + seq_printf(s,
> > > " SGT %p\n",
> > > + resource->import.sgt);
> > > + } else {
> > > + seq_printf(s,
> > > " SGT %p\n",
> > > + resource->alloc.sg_table);
> > > + }
> > > + seq_printf(s, " DMA_ADDR %pad\n",
> > > + &resource->dma_addr);
> > > + seq_printf(s,
> > > " VC_HANDLE %08x\n",
> > > + resource->vc_handle);
> > > + seq_printf(s, " VC_MAPPING %d\n",
> > > + resource->vpu_state);
> > > + }
> > > + }
> > > + seq_printf(s, "\n\nTotal resource count: %d\n\n",
> > > resource_count);
> > > +
> > > + mutex_unlock(&sm_state->map_lock);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/*
> > > + * Adds a buffer to the private data list which tracks all the
> > > allocated
> > > + * data.
> > > + */
> > > +static void vc_sm_add_resource(struct vc_sm_privdata_t *privdata,
> > > + struct vc_sm_buffer *buffer)
> > > +{
> > > + mutex_lock(&sm_state->map_lock);
> > > + list_add(&buffer->global_buffer_list, &sm_state->buffer_list);
> > > + mutex_unlock(&sm_state->map_lock);
> > > +
> > > + pr_debug("[%s]: added buffer %p (name %s, size %zu)\n",
> > > + __func__, buffer, buffer->name, buffer->size);
> > > +}
> > > +
> > > +/*
> > > + * Cleans up imported dmabuf.
> > > + */
> > > +static void vc_sm_clean_up_dmabuf(struct vc_sm_buffer *buffer)
> > > +{
> > > + if (!buffer->imported)
> > > + return;
> > > +
> > > + /* Handle cleaning up imported dmabufs */
> > > + mutex_lock(&buffer->lock);
> > > + if (buffer->import.sgt) {
> > > + dma_buf_unmap_attachment(buffer->import.attach,
> > > + buffer->import.sgt,
> > > + DMA_BIDIRECTIONAL);
> > > + buffer->import.sgt = NULL;
> > > + }
> > > + if (buffer->import.attach) {
> > > + dma_buf_detach(buffer->dma_buf, buffer->import.attach);
> > > + buffer->import.attach = NULL;
> > > + }
> > > + mutex_unlock(&buffer->lock);
> > > +}
> > > +
> > > +/*
> > > + * Instructs VPU to decrement the refcount on a buffer.
> > > + */
> > > +static void vc_sm_vpu_free(struct vc_sm_buffer *buffer)
> > > +{
> > > + if (buffer->vc_handle && buffer->vpu_state == VPU_MAPPED) {
> > > + struct vc_sm_free_t free = { buffer->vc_handle, 0 };
> > > + int status = vc_sm_cma_vchi_free(sm_state->sm_handle,
> > > &free,
> > > + &sm_state->int_trans_id);
> > > + if (status != 0 && status != -EINTR) {
> > > + pr_err("[%s]: failed to free memory on
> > > videocore (status: %u, trans_id: %u)\n",
> > > + __func__, status, sm_state-
> > > >int_trans_id);
> > > + }
> > > +
> > > + if (sm_state->require_released_callback) {
> > > + /* Need to wait for the VPU to confirm the
> > > free. */
> > > +
> > > + /* Retain a reference on this until the VPU has
> > > + * released it
> > > + */
> > > + buffer->vpu_state = VPU_UNMAPPING;
> > > + } else {
> > > + buffer->vpu_state = VPU_NOT_MAPPED;
> > > + buffer->vc_handle = 0;
> > > + }
> > > + }
> > > +}
> > > +
> > > +/*
> > > + * Release an allocation.
> > > + * All refcounting is done via the dma buf object.
> > > + *
> > > + * Must be called with the mutex held. The function will either
> > > release the
> > > + * mutex (if defering the release) or destroy it. The caller must
> > > therefore not
> > > + * reuse the buffer on return.
> > > + */
> > > +static void vc_sm_release_resource(struct vc_sm_buffer *buffer)
> > > +{
> > > + pr_debug("[%s]: buffer %p (name %s, size %zu), imported %u\n",
> > > + __func__, buffer, buffer->name, buffer->size,
> > > + buffer->imported);
> > > +
> > > + if (buffer->vc_handle) {
> > > + /* We've sent the unmap request but not had the
> > > response. */
> > > + pr_debug("[%s]: Waiting for VPU unmap response on
> > > %p\n",
> > > + __func__, buffer);
> > > + goto defer;
> > > + }
> > > + if (buffer->in_use) {
> > > + /* dmabuf still in use - we await the release */
> > > + pr_debug("[%s]: buffer %p is still in use\n", __func__,
> > > buffer);
> > > + goto defer;
> > > + }
> > > +
> > > + /* Release the allocation (whether imported dmabuf or CMA
> > > allocation) */
> > > + if (buffer->imported) {
> > > + if (buffer->import.dma_buf)
> > > + dma_buf_put(buffer->import.dma_buf);
> > > + else
> > > + pr_err("%s: Imported dmabuf already been put
> > > for buf %p\n",
> > > + __func__, buffer);
> > > + buffer->import.dma_buf = NULL;
> > > + } else {
> > > + dma_free_coherent(&sm_state->pdev->dev, buffer->size,
> > > + buffer->cookie, buffer->dma_addr);
> > > + }
> > > +
> > > + /* Free our buffer. Start by removing it from the list */
> > > + mutex_lock(&sm_state->map_lock);
> > > + list_del(&buffer->global_buffer_list);
> > > + mutex_unlock(&sm_state->map_lock);
> > > +
> > > + pr_debug("%s: Release our allocation - done\n", __func__);
> > > + mutex_unlock(&buffer->lock);
> > > +
> > > + mutex_destroy(&buffer->lock);
> > > +
> > > + kfree(buffer);
> > > + return;
> > > +
> > > +defer:
> > > + mutex_unlock(&buffer->lock);
> > > +}
> > > +
> > > +/* Create support for private data tracking. */
> > > +static struct vc_sm_privdata_t *vc_sm_cma_create_priv_data(pid_t id)
> > > +{
> > > + char alloc_name[32];
> > > + struct vc_sm_privdata_t *file_data = NULL;
> > > +
> > > + /* Allocate private structure. */
> > > + file_data = kzalloc(sizeof(*file_data), GFP_KERNEL);
> > > +
> > > + if (!file_data)
> > > + return NULL;
> > > +
> > > + snprintf(alloc_name, sizeof(alloc_name), "%d", id);
> > > +
> > > + file_data->pid = id;
> > > +
> > > + return file_data;
> > > +}
> > > +
> > > +/* Dma buf operations for use with our own allocations */
> > > +
> > > +static int vc_sm_dma_buf_attach(struct dma_buf *dmabuf,
> > > + struct dma_buf_attachment *attachment)
> > > +
> > > +{
> > > + struct vc_sm_dma_buf_attachment *a;
> > > + struct sg_table *sgt;
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > + struct scatterlist *rd, *wr;
> > > + int ret, i;
> > > +
> > > + a = kzalloc(sizeof(*a), GFP_KERNEL);
> > > + if (!a)
> > > + return -ENOMEM;
> > > +
> > > + pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf,
> > > attachment);
> > > +
> > > + mutex_lock(&buf->lock);
> > > +
> > > + INIT_LIST_HEAD(&a->list);
> > > +
> > > + sgt = &a->sg_table;
> > > +
> > > + /* Copy the buf->base_sgt scatter list to the attachment, as we
> > > can't
> > > + * map the same scatter list to multiple attachments at the
> > > same time.
> > > + */
> > > + ret = sg_alloc_table(sgt, buf->alloc.sg_table->orig_nents,
> > > GFP_KERNEL);
> > > + if (ret) {
> > > + kfree(a);
> > > + return -ENOMEM;
> > > + }
> > > +
> > > + rd = buf->alloc.sg_table->sgl;
> > > + wr = sgt->sgl;
> > > + for (i = 0; i < sgt->orig_nents; ++i) {
> > > + sg_set_page(wr, sg_page(rd), rd->length, rd->offset);
> > > + rd = sg_next(rd);
> > > + wr = sg_next(wr);
> > > + }
> > > +
> > > + a->dma_dir = DMA_NONE;
> > > + attachment->priv = a;
> > > +
> > > + list_add(&a->list, &buf->attachments);
> > > + mutex_unlock(&buf->lock);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static void vc_sm_dma_buf_detach(struct dma_buf *dmabuf,
> > > + struct dma_buf_attachment *attachment)
> > > +{
> > > + struct vc_sm_dma_buf_attachment *a = attachment->priv;
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > + struct sg_table *sgt;
> > > +
> > > + pr_debug("%s dmabuf %p attachment %p\n", __func__, dmabuf,
> > > attachment);
> > > + if (!a)
> > > + return;
> > > +
> > > + sgt = &a->sg_table;
> > > +
> > > + /* release the scatterlist cache */
> > > + if (a->dma_dir != DMA_NONE)
> > > + dma_unmap_sg(attachment->dev, sgt->sgl, sgt-
> > > >orig_nents,
> > > + a->dma_dir);
> > > + sg_free_table(sgt);
> > > +
> > > + mutex_lock(&buf->lock);
> > > + list_del(&a->list);
> > > + mutex_unlock(&buf->lock);
> > > +
> > > + kfree(a);
> > > +}
> > > +
> > > +static struct sg_table *vc_sm_map_dma_buf(struct dma_buf_attachment
> > > *attachment,
> > > + enum dma_data_direction
> > > direction)
> > > +{
> > > + struct vc_sm_dma_buf_attachment *a = attachment->priv;
> > > + /* stealing dmabuf mutex to serialize map/unmap operations */
> > > + struct mutex *lock = &attachment->dmabuf->lock;
> > > + struct sg_table *table;
> > > +
> > > + mutex_lock(lock);
> > > + pr_debug("%s attachment %p\n", __func__, attachment);
> > > + table = &a->sg_table;
> > > +
> > > + /* return previously mapped sg table */
> > > + if (a->dma_dir == direction) {
> > > + mutex_unlock(lock);
> > > + return table;
> > > + }
> > > +
> > > + /* release any previous cache */
> > > + if (a->dma_dir != DMA_NONE) {
> > > + dma_unmap_sg(attachment->dev, table->sgl, table-
> > > >orig_nents,
> > > + a->dma_dir);
> > > + a->dma_dir = DMA_NONE;
> > > + }
> > > +
> > > + /* mapping to the client with new direction */
> > > + table->nents = dma_map_sg(attachment->dev, table->sgl,
> > > + table->orig_nents, direction);
> > > + if (!table->nents) {
> > > + pr_err("failed to map scatterlist\n");
> > > + mutex_unlock(lock);
> > > + return ERR_PTR(-EIO);
> > > + }
> > > +
> > > + a->dma_dir = direction;
> > > + mutex_unlock(lock);
> > > +
> > > + pr_debug("%s attachment %p\n", __func__, attachment);
> > > + return table;
> > > +}
> > > +
> > > +static void vc_sm_unmap_dma_buf(struct dma_buf_attachment
> > > *attachment,
> > > + struct sg_table *table,
> > > + enum dma_data_direction direction)
> > > +{
> > > + pr_debug("%s attachment %p\n", __func__, attachment);
> > > + dma_unmap_sg(attachment->dev, table->sgl, table->nents,
> > > direction);
> > > +}
> > > +
> > > +static int vc_sm_dmabuf_mmap(struct dma_buf *dmabuf, struct
> > > vm_area_struct *vma)
> > > +{
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > + int ret;
> > > +
> > > + pr_debug("%s dmabuf %p, buf %p, vm_start %08lX\n", __func__,
> > > dmabuf,
> > > + buf, vma->vm_start);
> > > +
> > > + mutex_lock(&buf->lock);
> > > +
> > > + /* now map it to userspace */
> > > + vma->vm_pgoff = 0;
> > > +
> > > + ret = dma_mmap_coherent(&sm_state->pdev->dev, vma, buf->cookie,
> > > + buf->dma_addr, buf->size);
> > > +
> > > + if (ret) {
> > > + pr_err("Remapping memory failed, error: %d\n", ret);
> > > + return ret;
> > > + }
> > > +
> > > + vma->vm_flags |= VM_DONTEXPAND | VM_DONTDUMP;
> > > +
> > > + mutex_unlock(&buf->lock);
> > > +
> > > + if (ret)
> > > + pr_err("%s: failure mapping buffer to userspace\n",
> > > + __func__);
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +static void vc_sm_dma_buf_release(struct dma_buf *dmabuf)
> > > +{
> > > + struct vc_sm_buffer *buffer;
> > > +
> > > + if (!dmabuf)
> > > + return;
> > > +
> > > + buffer = (struct vc_sm_buffer *)dmabuf->priv;
> > > +
> > > + mutex_lock(&buffer->lock);
> > > +
> > > + pr_debug("%s dmabuf %p, buffer %p\n", __func__, dmabuf,
> > > buffer);
> > > +
> > > + buffer->in_use = 0;
> > > +
> > > + /* Unmap on the VPU */
> > > + vc_sm_vpu_free(buffer);
> > > + pr_debug("%s vpu_free done\n", __func__);
> > > +
> > > + /* Unmap our dma_buf object (the vc_sm_buffer remains until
> > > released
> > > + * on the VPU).
> > > + */
> > > + vc_sm_clean_up_dmabuf(buffer);
> > > + pr_debug("%s clean_up dmabuf done\n", __func__);
> > > +
> > > + /* buffer->lock will be destroyed by vc_sm_release_resource if
> > > finished
> > > + * with, otherwise unlocked. Do NOT unlock here.
> > > + */
> > > + vc_sm_release_resource(buffer);
> > > + pr_debug("%s done\n", __func__);
> > > +}
> > > +
> > > +static int vc_sm_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> > > + enum dma_data_direction
> > > direction)
> > > +{
> > > + struct vc_sm_buffer *buf;
> > > + struct vc_sm_dma_buf_attachment *a;
> > > +
> > > + if (!dmabuf)
> > > + return -EFAULT;
> > > +
> > > + buf = dmabuf->priv;
> > > + if (!buf)
> > > + return -EFAULT;
> > > +
> > > + mutex_lock(&buf->lock);
> > > +
> > > + list_for_each_entry(a, &buf->attachments, list) {
> > > + dma_sync_sg_for_cpu(a->dev, a->sg_table.sgl,
> > > + a->sg_table.nents, direction);
> > > + }
> > > + mutex_unlock(&buf->lock);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int vc_sm_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> > > + enum dma_data_direction
> > > direction)
> > > +{
> > > + struct vc_sm_buffer *buf;
> > > + struct vc_sm_dma_buf_attachment *a;
> > > +
> > > + if (!dmabuf)
> > > + return -EFAULT;
> > > + buf = dmabuf->priv;
> > > + if (!buf)
> > > + return -EFAULT;
> > > +
> > > + mutex_lock(&buf->lock);
> > > +
> > > + list_for_each_entry(a, &buf->attachments, list) {
> > > + dma_sync_sg_for_device(a->dev, a->sg_table.sgl,
> > > + a->sg_table.nents, direction);
> > > + }
> > > + mutex_unlock(&buf->lock);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static const struct dma_buf_ops dma_buf_ops = {
> > > + .map_dma_buf = vc_sm_map_dma_buf,
> > > + .unmap_dma_buf = vc_sm_unmap_dma_buf,
> > > + .mmap = vc_sm_dmabuf_mmap,
> > > + .release = vc_sm_dma_buf_release,
> > > + .attach = vc_sm_dma_buf_attach,
> > > + .detach = vc_sm_dma_buf_detach,
> > > + .begin_cpu_access = vc_sm_dma_buf_begin_cpu_access,
> > > + .end_cpu_access = vc_sm_dma_buf_end_cpu_access,
> > > +};
> > > +
> > > +/* Dma_buf operations for chaining through to an imported dma_buf */
> > > +
> > > +static
> > > +int vc_sm_import_dma_buf_attach(struct dma_buf *dmabuf,
> > > + struct dma_buf_attachment *attachment)
> > > +{
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > +
> > > + if (!buf->imported)
> > > + return -EINVAL;
> > > + return buf->import.dma_buf->ops->attach(buf->import.dma_buf,
> > > + attachment);
> > > +}
> > > +
> > > +static
> > > +void vc_sm_import_dma_buf_detatch(struct dma_buf *dmabuf,
> > > + struct dma_buf_attachment
> > > *attachment)
> > > +{
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > +
> > > + if (!buf->imported)
> > > + return;
> > > + buf->import.dma_buf->ops->detach(buf->import.dma_buf,
> > > attachment);
> > > +}
> > > +
> > > +static
> > > +struct sg_table *vc_sm_import_map_dma_buf(struct dma_buf_attachment
> > > *attachment,
> > > + enum dma_data_direction
> > > direction)
> > > +{
> > > + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> > > +
> > > + if (!buf->imported)
> > > + return NULL;
> > > + return buf->import.dma_buf->ops->map_dma_buf(attachment,
> > > + direction);
> > > +}
> > > +
> > > +static
> > > +void vc_sm_import_unmap_dma_buf(struct dma_buf_attachment
> > > *attachment,
> > > + struct sg_table *table,
> > > + enum dma_data_direction direction)
> > > +{
> > > + struct vc_sm_buffer *buf = attachment->dmabuf->priv;
> > > +
> > > + if (!buf->imported)
> > > + return;
> > > + buf->import.dma_buf->ops->unmap_dma_buf(attachment, table,
> > > direction);
> > > +}
> > > +
> > > +static
> > > +int vc_sm_import_dmabuf_mmap(struct dma_buf *dmabuf, struct
> > > vm_area_struct *vma)
> > > +{
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > +
> > > + pr_debug("%s: mmap dma_buf %p, buf %p, imported db %p\n",
> > > __func__,
> > > + dmabuf, buf, buf->import.dma_buf);
> > > + if (!buf->imported) {
> > > + pr_err("%s: mmap dma_buf %p- not an imported buffer\n",
> > > + __func__, dmabuf);
> > > + return -EINVAL;
> > > + }
> > > + return buf->import.dma_buf->ops->mmap(buf->import.dma_buf,
> > > vma);
> > > +}
> > > +
> > > +static
> > > +void vc_sm_import_dma_buf_release(struct dma_buf *dmabuf)
> > > +{
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > +
> > > + pr_debug("%s: Relasing dma_buf %p\n", __func__, dmabuf);
> > > + mutex_lock(&buf->lock);
> > > + if (!buf->imported)
> > > + return;
> > > +
> > > + buf->in_use = 0;
> > > +
> > > + vc_sm_vpu_free(buf);
> > > +
> > > + vc_sm_release_resource(buf);
> > > +}
> > > +
> > > +static
> > > +int vc_sm_import_dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
> > > + enum dma_data_direction
> > > direction)
> > > +{
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > +
> > > + if (!buf->imported)
> > > + return -EINVAL;
> > > + return buf->import.dma_buf->ops->begin_cpu_access(buf-
> > > >import.dma_buf,
> > > + direction);
> > > +}
> > > +
> > > +static
> > > +int vc_sm_import_dma_buf_end_cpu_access(struct dma_buf *dmabuf,
> > > + enum dma_data_direction
> > > direction)
> > > +{
> > > + struct vc_sm_buffer *buf = dmabuf->priv;
> > > +
> > > + if (!buf->imported)
> > > + return -EINVAL;
> > > + return buf->import.dma_buf->ops->end_cpu_access(buf-
> > > >import.dma_buf,
> > > + direction);
> > > +}
> > > +
> > > +static const struct dma_buf_ops dma_buf_import_ops = {
> > > + .map_dma_buf = vc_sm_import_map_dma_buf,
> > > + .unmap_dma_buf = vc_sm_import_unmap_dma_buf,
> > > + .mmap = vc_sm_import_dmabuf_mmap,
> > > + .release = vc_sm_import_dma_buf_release,
> > > + .attach = vc_sm_import_dma_buf_attach,
> > > + .detach = vc_sm_import_dma_buf_detatch,
> > > + .begin_cpu_access = vc_sm_import_dma_buf_begin_cpu_access,
> > > + .end_cpu_access = vc_sm_import_dma_buf_end_cpu_access,
> > > +};
> > > +
> > > +/* Import a dma_buf to be shared with VC. */
> > > +int
> > > +vc_sm_cma_import_dmabuf_internal(struct vc_sm_privdata_t *private,
> > > + struct dma_buf *dma_buf,
> > > + int fd,
> > > + struct dma_buf **imported_buf)
> > > +{
> > > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> > > + struct vc_sm_buffer *buffer = NULL;
> > > + struct vc_sm_import import = { };
> > > + struct vc_sm_import_result result = { };
> > > + struct dma_buf_attachment *attach = NULL;
> > > + struct sg_table *sgt = NULL;
> > > + dma_addr_t dma_addr;
> > > + int ret = 0;
> > > + int status;
> > > +
> > > + /* Setup our allocation parameters */
> > > + pr_debug("%s: importing dma_buf %p/fd %d\n", __func__, dma_buf,
> > > fd);
> > > +
> > > + if (fd < 0)
> > > + get_dma_buf(dma_buf);
> > > + else
> > > + dma_buf = dma_buf_get(fd);
> > > +
> > > + if (!dma_buf)
> > > + return -EINVAL;
> > > +
> > > + attach = dma_buf_attach(dma_buf, &sm_state->pdev->dev);
> > > + if (IS_ERR(attach)) {
> > > + ret = PTR_ERR(attach);
> > > + goto error;
> > > + }
> > > +
> > > + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> > > + if (IS_ERR(sgt)) {
> > > + ret = PTR_ERR(sgt);
> > > + goto error;
> > > + }
> > > +
> > > + /* Verify that the address block is contiguous */
> > > + if (sgt->nents != 1) {
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + /* Allocate local buffer to track this allocation. */
> > > + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> > > + if (!buffer) {
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + import.type = VC_SM_ALLOC_NON_CACHED;
> > > + dma_addr = sg_dma_address(sgt->sgl);
> > > + import.addr = (u32)dma_addr;
> > > + if ((import.addr & 0xC0000000) != 0xC0000000) {
> > > + pr_err("%s: Expecting an uncached alias for dma_addr
> > > %pad\n",
> > > + __func__, &dma_addr);
> > > + import.addr |= 0xC0000000;
> > > + }
> > > + import.size = sg_dma_len(sgt->sgl);
> > > + import.allocator = current->tgid;
> > > + import.kernel_id = get_kernel_id(buffer);
> > > +
> > > + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> > > + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> > > +
> > > + pr_debug("[%s]: attempt to import \"%s\" data - type %u, addr
> > > %pad, size %u.\n",
> > > + __func__, import.name, import.type, &dma_addr,
> > > import.size);
> > > +
> > > + /* Allocate the videocore buffer. */
> > > + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import,
> > > &result,
> > > + &sm_state->int_trans_id);
> > > + if (status == -EINTR) {
> > > + pr_debug("[%s]: requesting import memory action restart
> > > (trans_id: %u)\n",
> > > + __func__, sm_state->int_trans_id);
> > > + ret = -ERESTARTSYS;
> > > + private->restart_sys = -EINTR;
> > > + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> > > + goto error;
> > > + } else if (status || !result.res_handle) {
> > > + pr_debug("[%s]: failed to import memory on videocore
> > > (status: %u, trans_id: %u)\n",
> > > + __func__, status, sm_state->int_trans_id);
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + mutex_init(&buffer->lock);
> > > + INIT_LIST_HEAD(&buffer->attachments);
> > > + memcpy(buffer->name, import.name,
> > > + min(sizeof(buffer->name), sizeof(import.name) - 1));
> > > +
> > > + /* Keep track of the buffer we created. */
> > > + buffer->private = private;
> > > + buffer->vc_handle = result.res_handle;
> > > + buffer->size = import.size;
> > > + buffer->vpu_state = VPU_MAPPED;
> > > +
> > > + buffer->imported = 1;
> > > + buffer->import.dma_buf = dma_buf;
> > > +
> > > + buffer->import.attach = attach;
> > > + buffer->import.sgt = sgt;
> > > + buffer->dma_addr = dma_addr;
> > > + buffer->in_use = 1;
> > > + buffer->kernel_id = import.kernel_id;
> > > +
> > > + /*
> > > + * We're done - we need to export a new dmabuf chaining through
> > > most
> > > + * functions, but enabling us to release our own internal
> > > references
> > > + * here.
> > > + */
> > > + exp_info.ops = &dma_buf_import_ops;
> > > + exp_info.size = import.size;
> > > + exp_info.flags = O_RDWR;
> > > + exp_info.priv = buffer;
> > > +
> > > + buffer->dma_buf = dma_buf_export(&exp_info);
> > > + if (IS_ERR(buffer->dma_buf)) {
> > > + ret = PTR_ERR(buffer->dma_buf);
> > > + goto error;
> > > + }
> > > +
> > > + vc_sm_add_resource(private, buffer);
> > > +
> > > + *imported_buf = buffer->dma_buf;
> > > +
> > > + return 0;
> > > +
> > > +error:
> > > + if (result.res_handle) {
> > > + struct vc_sm_free_t free = { result.res_handle, 0 };
> > > +
> > > + vc_sm_cma_vchi_free(sm_state->sm_handle, &free,
> > > + &sm_state->int_trans_id);
> > > + }
> > > + free_kernel_id(import.kernel_id);
> > > + kfree(buffer);
> > > + if (sgt)
> > > + dma_buf_unmap_attachment(attach, sgt,
> > > DMA_BIDIRECTIONAL);
> > > + if (attach)
> > > + dma_buf_detach(dma_buf, attach);
> > > + dma_buf_put(dma_buf);
> > > + return ret;
> > > +}
> > > +
> > > +static int vc_sm_cma_vpu_alloc(u32 size, u32 align, const char
> > > *name,
> > > + u32 mem_handle, struct vc_sm_buffer
> > > **ret_buffer)
> > > +{
> > > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> > > + struct vc_sm_buffer *buffer = NULL;
> > > + struct sg_table *sgt;
> > > + int aligned_size;
> > > + int ret = 0;
> > > +
> > > + /* Align to the user requested align */
> > > + aligned_size = ALIGN(size, align);
> > > + /* and then to a page boundary */
> > > + aligned_size = PAGE_ALIGN(aligned_size);
> > > +
> > > + if (!aligned_size)
> > > + return -EINVAL;
> > > +
> > > + /* Allocate local buffer to track this allocation. */
> > > + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> > > + if (!buffer)
> > > + return -ENOMEM;
> > > +
> > > + mutex_init(&buffer->lock);
> > > + /* Acquire the mutex as vc_sm_release_resource will release it
> > > in the
> > > + * error path.
> > > + */
> > > + mutex_lock(&buffer->lock);
> > > +
> > > + buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
> > > + aligned_size, &buffer-
> > > >dma_addr,
> > > + GFP_KERNEL);
> > > + if (!buffer->cookie) {
> > > + pr_err("[%s]: dma_alloc_coherent alloc of %d bytes
> > > failed\n",
> > > + __func__, aligned_size);
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + pr_debug("[%s]: alloc of %d bytes success\n",
> > > + __func__, aligned_size);
> > > +
> > > + sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
> > > + if (!sgt) {
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer-
> > > >cookie,
> > > + buffer->dma_addr, buffer->size);
> > > + if (ret < 0) {
> > > + pr_err("failed to get scatterlist from DMA API\n");
> > > + kfree(sgt);
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > + buffer->alloc.sg_table = sgt;
> > > +
> > > + INIT_LIST_HEAD(&buffer->attachments);
> > > +
> > > + memcpy(buffer->name, name,
> > > + min(sizeof(buffer->name), strlen(name)));
> > > +
> > > + exp_info.ops = &dma_buf_ops;
> > > + exp_info.size = aligned_size;
> > > + exp_info.flags = O_RDWR;
> > > + exp_info.priv = buffer;
> > > +
> > > + buffer->dma_buf = dma_buf_export(&exp_info);
> > > + if (IS_ERR(buffer->dma_buf)) {
> > > + ret = PTR_ERR(buffer->dma_buf);
> > > + goto error;
> > > + }
> > > + buffer->dma_addr = (u32)sg_dma_address(buffer->alloc.sg_table-
> > > >sgl);
> > > + if ((buffer->dma_addr & 0xC0000000) != 0xC0000000) {
> > > + pr_warn_once("%s: Expecting an uncached alias for
> > > dma_addr %pad\n",
> > > + __func__, &buffer->dma_addr);
> > > + buffer->dma_addr |= 0xC0000000;
> > > + }
> > > + buffer->private = sm_state->vpu_allocs;
> > > +
> > > + buffer->vc_handle = mem_handle;
> > > + buffer->vpu_state = VPU_MAPPED;
> > > + buffer->vpu_allocated = 1;
> > > + buffer->size = size;
> > > + /*
> > > + * Create an ID that will be passed along with our message so
> > > + * that when we service the release reply, we can look up which
> > > + * resource is being released.
> > > + */
> > > + buffer->kernel_id = get_kernel_id(buffer);
> > > +
> > > + vc_sm_add_resource(sm_state->vpu_allocs, buffer);
> > > +
> > > + mutex_unlock(&buffer->lock);
> > > +
> > > + *ret_buffer = buffer;
> > > + return 0;
> > > +error:
> > > + if (buffer)
> > > + vc_sm_release_resource(buffer);
> > > + return ret;
> > > +}
> > > +
> > > +static void
> > > +vc_sm_vpu_event(struct sm_instance *instance, struct vc_sm_result_t
> > > *reply,
> > > + int reply_len)
> > > +{
> > > + switch (reply->trans_id & ~0x80000000) {
> > > + case VC_SM_MSG_TYPE_CLIENT_VERSION:
> > > + {
> > > + /* Acknowledge that the firmware supports the version
> > > command */
> > > + pr_debug("%s: firmware acked version msg. Require
> > > release cb\n",
> > > + __func__);
> > > + sm_state->require_released_callback = true;
> > > + }
> > > + break;
> > > + case VC_SM_MSG_TYPE_RELEASED:
> > > + {
> > > + struct vc_sm_released *release = (struct vc_sm_released
> > > *)reply;
> > > + struct vc_sm_buffer *buffer =
> > > + lookup_kernel_id(release-
> > > >kernel_id);
> > > + if (!buffer) {
> > > + pr_err("%s: VC released a buffer that is
> > > already released, kernel_id %d\n",
> > > + __func__, release->kernel_id);
> > > + break;
> > > + }
> > > + mutex_lock(&buffer->lock);
> > > +
> > > + pr_debug("%s: Released addr %08x, size %u, id %08x,
> > > mem_handle %08x\n",
> > > + __func__, release->addr, release->size,
> > > + release->kernel_id, release->vc_handle);
> > > +
> > > + buffer->vc_handle = 0;
> > > + buffer->vpu_state = VPU_NOT_MAPPED;
> > > + free_kernel_id(release->kernel_id);
> > > +
> > > + if (buffer->vpu_allocated) {
> > > + /* VPU allocation, so release the dmabuf which
> > > will
> > > + * trigger the clean up.
> > > + */
> > > + mutex_unlock(&buffer->lock);
> > > + dma_buf_put(buffer->dma_buf);
> > > + } else {
> > > + vc_sm_release_resource(buffer);
> > > + }
> > > + }
> > > + break;
> > > + case VC_SM_MSG_TYPE_VC_MEM_REQUEST:
> > > + {
> > > + struct vc_sm_buffer *buffer = NULL;
> > > + struct vc_sm_vc_mem_request *req =
> > > + (struct vc_sm_vc_mem_request
> > > *)reply;
> > > + struct vc_sm_vc_mem_request_result reply;
> > > + int ret;
> > > +
> > > + pr_debug("%s: Request %u bytes of memory, align %d name
> > > %s, trans_id %08x\n",
> > > + __func__, req->size, req->align, req->name,
> > > + req->trans_id);
> > > + ret = vc_sm_cma_vpu_alloc(req->size, req->align, req-
> > > >name,
> > > + req->vc_handle, &buffer);
> > > +
> > > + reply.trans_id = req->trans_id;
> > > + if (!ret) {
> > > + reply.addr = buffer->dma_addr;
> > > + reply.kernel_id = buffer->kernel_id;
> > > + pr_debug("%s: Allocated resource buffer %p,
> > > addr %pad\n",
> > > + __func__, buffer, &buffer->dma_addr);
> > > + } else {
> > > + pr_err("%s: Allocation failed size %u, name %s,
> > > vc_handle %u\n",
> > > + __func__, req->size, req->name, req-
> > > >vc_handle);
> > > + reply.addr = 0;
> > > + reply.kernel_id = 0;
> > > + }
> > > + vc_sm_vchi_client_vc_mem_req_reply(sm_state->sm_handle,
> > > &reply,
> > > + &sm_state-
> > > >int_trans_id);
> > > + break;
> > > + }
> > > + break;
> > > + default:
> > > + pr_err("%s: Unknown vpu cmd %x\n", __func__, reply-
> > > >trans_id);
> > > + break;
> > > + }
> > > +}
> > > +
> > > +/* Userspace handling */
> > > +/*
> > > + * Open the device. Creates a private state to help track all
> > > allocation
> > > + * associated with this device.
> > > + */
> > > +static int vc_sm_cma_open(struct inode *inode, struct file *file)
> > > +{
> > > + /* Make sure the device was started properly. */
> > > + if (!sm_state) {
> > > + pr_err("[%s]: invalid device\n", __func__);
> > > + return -EPERM;
> > > + }
> > > +
> > > + file->private_data = vc_sm_cma_create_priv_data(current->tgid);
> > > + if (!file->private_data) {
> > > + pr_err("[%s]: failed to create data tracker\n",
> > > __func__);
> > > +
> > > + return -ENOMEM;
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/*
> > > + * Close the vcsm-cma device.
> > > + * All allocations are file descriptors to the dmabuf objects, so we
> > > will get
> > > + * the clean up request on those as those are cleaned up.
> > > + */
> > > +static int vc_sm_cma_release(struct inode *inode, struct file *file)
> > > +{
> > > + struct vc_sm_privdata_t *file_data =
> > > + (struct vc_sm_privdata_t *)file->private_data;
> > > + int ret = 0;
> > > +
> > > + /* Make sure the device was started properly. */
> > > + if (!sm_state || !file_data) {
> > > + pr_err("[%s]: invalid device\n", __func__);
> > > + ret = -EPERM;
> > > + goto out;
> > > + }
> > > +
> > > + pr_debug("[%s]: using private data %p\n", __func__, file_data);
> > > +
> > > + /* Terminate the private data. */
> > > + kfree(file_data);
> > > +
> > > +out:
> > > + return ret;
> > > +}
> > > +
> > > +/*
> > > + * Allocate a shared memory handle and block.
> > > + * Allocation is from CMA, and then imported into the VPU mappings.
> > > + */
> > > +int vc_sm_cma_ioctl_alloc(struct vc_sm_privdata_t *private,
> > > + struct vc_sm_cma_ioctl_alloc *ioparam)
> > > +{
> > > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> > > + struct vc_sm_buffer *buffer = NULL;
> > > + struct vc_sm_import import = { 0 };
> > > + struct vc_sm_import_result result = { 0 };
> > > + struct dma_buf *dmabuf = NULL;
> > > + struct sg_table *sgt;
> > > + int aligned_size;
> > > + int ret = 0;
> > > + int status;
> > > + int fd = -1;
> > > +
> > > + aligned_size = PAGE_ALIGN(ioparam->size);
> > > +
> > > + if (!aligned_size)
> > > + return -EINVAL;
> > > +
> > > + /* Allocate local buffer to track this allocation. */
> > > + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> > > + if (!buffer) {
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + buffer->cookie = dma_alloc_coherent(&sm_state->pdev->dev,
> > > + aligned_size,
> > > + &buffer->dma_addr,
> > > + GFP_KERNEL);
> > > + if (!buffer->cookie) {
> > > + pr_err("[%s]: dma_alloc_coherent alloc of %d bytes
> > > failed\n",
> > > + __func__, aligned_size);
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + import.type = VC_SM_ALLOC_NON_CACHED;
> > > + import.allocator = current->tgid;
> > > +
> > > + if (*ioparam->name)
> > > + memcpy(import.name, ioparam->name, sizeof(import.name)
> > > - 1);
> > > + else
> > > + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> > > + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> > > +
> > > + mutex_init(&buffer->lock);
> > > + INIT_LIST_HEAD(&buffer->attachments);
> > > + memcpy(buffer->name, import.name,
> > > + min(sizeof(buffer->name), sizeof(import.name) - 1));
> > > +
> > > + exp_info.ops = &dma_buf_ops;
> > > + exp_info.size = aligned_size;
> > > + exp_info.flags = O_RDWR;
> > > + exp_info.priv = buffer;
> > > +
> > > + dmabuf = dma_buf_export(&exp_info);
> > > + if (IS_ERR(dmabuf)) {
> > > + ret = PTR_ERR(dmabuf);
> > > + goto error;
> > > + }
> > > + buffer->dma_buf = dmabuf;
> > > +
> > > + import.addr = buffer->dma_addr;
> > > + import.size = aligned_size;
> > > + import.kernel_id = get_kernel_id(buffer);
> > > +
> > > + /* Wrap it into a videocore buffer. */
> > > + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import,
> > > &result,
> > > + &sm_state->int_trans_id);
> > > + if (status == -EINTR) {
> > > + pr_debug("[%s]: requesting import memory action restart
> > > (trans_id: %u)\n",
> > > + __func__, sm_state->int_trans_id);
> > > + ret = -ERESTARTSYS;
> > > + private->restart_sys = -EINTR;
> > > + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> > > + goto error;
> > > + } else if (status || !result.res_handle) {
> > > + pr_err("[%s]: failed to import memory on videocore
> > > (status: %u, trans_id: %u)\n",
> > > + __func__, status, sm_state->int_trans_id);
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + /* Keep track of the buffer we created. */
> > > + buffer->private = private;
> > > + buffer->vc_handle = result.res_handle;
> > > + buffer->size = import.size;
> > > + buffer->vpu_state = VPU_MAPPED;
> > > + buffer->kernel_id = import.kernel_id;
> > > +
> > > + sgt = kmalloc(sizeof(*sgt), GFP_KERNEL);
> > > + if (!sgt) {
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > +
> > > + ret = dma_get_sgtable(&sm_state->pdev->dev, sgt, buffer-
> > > >cookie,
> > > + buffer->dma_addr, buffer->size);
> > > + if (ret < 0) {
> > > + /* FIXME: error handling */
> > > + pr_err("failed to get scatterlist from DMA API\n");
> > > + kfree(sgt);
> > > + ret = -ENOMEM;
> > > + goto error;
> > > + }
> > > + buffer->alloc.sg_table = sgt;
> > > +
> > > + fd = dma_buf_fd(dmabuf, O_CLOEXEC);
> > > + if (fd < 0)
> > > + goto error;
> > > +
> > > + vc_sm_add_resource(private, buffer);
> > > +
> > > + pr_debug("[%s]: Added resource as fd %d, buffer %p, private %p,
> > > dma_addr %pad\n",
> > > + __func__, fd, buffer, private, &buffer->dma_addr);
> > > +
> > > + /* We're done */
> > > + ioparam->handle = fd;
> > > + ioparam->vc_handle = buffer->vc_handle;
> > > + ioparam->dma_addr = buffer->dma_addr;
> > > + return 0;
> > > +
> > > +error:
> > > + pr_err("[%s]: something failed - cleanup. ret %d\n", __func__,
> > > ret);
> > > +
> > > + if (dmabuf) {
> > > + /* dmabuf has been exported, therefore allow dmabuf
> > > cleanup to
> > > + * deal with this
> > > + */
> > > + dma_buf_put(dmabuf);
> > > + } else {
> > > + /* No dmabuf, therefore just free the buffer here */
> > > + if (buffer->cookie)
> > > + dma_free_coherent(&sm_state->pdev->dev, buffer-
> > > >size,
> > > + buffer->cookie, buffer-
> > > >dma_addr);
> > > + kfree(buffer);
> > > + }
> > > + return ret;
> > > +}
> > > +
> > > +#ifndef CONFIG_ARM64
> > > +/* Converts VCSM_CACHE_OP_* to an operating function. */
> > > +static void (*cache_op_to_func(const unsigned int cache_op))
> > > + (const void*, const
> > > void*)
> > > +{
> > > + switch (cache_op) {
> > > + case VC_SM_CACHE_OP_NOP:
> > > + return NULL;
> > > +
> > > + case VC_SM_CACHE_OP_INV:
> > > + case VC_SM_CACHE_OP_CLEAN:
> > > + case VC_SM_CACHE_OP_FLUSH:
> > > + return dmac_flush_range;
> > > +
> > > + default:
> > > + pr_err("[%s]: Invalid cache_op: 0x%08x\n", __func__,
> > > cache_op);
> > > + return NULL;
> > > + }
> > > +}
> > > +
> > > +/*
> > > + * Clean/invalid/flush cache of which buffer is already pinned (i.e.
> > > accessed).
> > > + */
> > > +static int clean_invalid_contig_2d(const void __user *addr,
> > > + const size_t block_count,
> > > + const size_t block_size,
> > > + const size_t stride,
> > > + const unsigned int cache_op)
> > > +{
> > > + size_t i;
> > > + void (*op_fn)(const void *start, const void *end);
> > > +
> > > + if (!block_size) {
> > > + pr_err("[%s]: size cannot be 0\n", __func__);
> > > + return -EINVAL;
> > > + }
> > > +
> > > + op_fn = cache_op_to_func(cache_op);
> > > + if (!op_fn)
> > > + return -EINVAL;
> > > +
> > > + for (i = 0; i < block_count; i ++, addr += stride)
> > > + op_fn(addr, addr + block_size);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static int vc_sm_cma_clean_invalid2(unsigned int cmdnr, unsigned
> > > long arg)
> > > +{
> > > + struct vc_sm_cma_ioctl_clean_invalid2 ioparam;
> > > + struct vc_sm_cma_ioctl_clean_invalid_block *block = NULL;
> > > + int i, ret = 0;
> > > +
> > > + /* Get parameter data. */
> > > + if (copy_from_user(&ioparam, (void *)arg, sizeof(ioparam))) {
> > > + pr_err("[%s]: failed to copy-from-user header for cmd
> > > %x\n",
> > > + __func__, cmdnr);
> > > + return -EFAULT;
> > > + }
> > > + block = kmalloc(ioparam.op_count * sizeof(*block), GFP_KERNEL);
> > > + if (!block)
> > > + return -EFAULT;
> > > +
> > > + if (copy_from_user(block, (void *)(arg + sizeof(ioparam)),
> > > + ioparam.op_count * sizeof(*block)) != 0) {
> > > + pr_err("[%s]: failed to copy-from-user payload for cmd
> > > %x\n",
> > > + __func__, cmdnr);
> > > + ret = -EFAULT;
> > > + goto out;
> > > + }
> > > +
> > > + for (i = 0; i < ioparam.op_count; i++) {
> > > + const struct vc_sm_cma_ioctl_clean_invalid_block *
> > > const op =
> > > + block +
> > > i;
> > > +
> > > + if (op->invalidate_mode == VC_SM_CACHE_OP_NOP)
> > > + continue;
> > > +
> > > + ret = clean_invalid_contig_2d((void __user *)op-
> > > >start_address,
> > > + op->block_count, op-
> > > >block_size,
> > > + op->inter_block_stride,
> > > + op->invalidate_mode);
> > > + if (ret)
> > > + break;
> > > + }
> > > +out:
> > > + kfree(block);
> > > +
> > > + return ret;
> > > +}
> > > +#endif
> > > +
> > > +static long vc_sm_cma_ioctl(struct file *file, unsigned int cmd,
> > > + unsigned long arg)
> > > +{
> > > + int ret = 0;
> > > + unsigned int cmdnr = _IOC_NR(cmd);
> > > + struct vc_sm_privdata_t *file_data =
> > > + (struct vc_sm_privdata_t *)file->private_data;
> > > +
> > > + /* Validate we can work with this device. */
> > > + if (!sm_state || !file_data) {
> > > + pr_err("[%s]: invalid device\n", __func__);
> > > + return -EPERM;
> > > + }
> > > +
> > > + /* Action is a re-post of a previously interrupted action? */
> > > + if (file_data->restart_sys == -EINTR) {
> > > + struct vc_sm_action_clean_t action_clean;
> > > +
> > > + pr_debug("[%s]: clean up of action %u (trans_id: %u)
> > > following EINTR\n",
> > > + __func__, file_data->int_action,
> > > + file_data->int_trans_id);
> > > +
> > > + action_clean.res_action = file_data->int_action;
> > > + action_clean.action_trans_id = file_data->int_trans_id;
> > > +
> > > + file_data->restart_sys = 0;
> > > + }
> > > +
> > > + /* Now process the command. */
> > > + switch (cmdnr) {
> > > + /* New memory allocation.
> > > + */
> > > + case VC_SM_CMA_CMD_ALLOC:
> > > + {
> > > + struct vc_sm_cma_ioctl_alloc ioparam;
> > > +
> > > + /* Get the parameter data. */
> > > + if (copy_from_user
> > > + (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
> > > + pr_err("[%s]: failed to copy-from-user for cmd
> > > %x\n",
> > > + __func__, cmdnr);
> > > + ret = -EFAULT;
> > > + break;
> > > + }
> > > +
> > > + ret = vc_sm_cma_ioctl_alloc(file_data, &ioparam);
> > > + if (!ret &&
> > > + (copy_to_user((void *)arg, &ioparam,
> > > + sizeof(ioparam)) != 0)) {
> > > + /* FIXME: Release allocation */
> > > + pr_err("[%s]: failed to copy-to-user for cmd
> > > %x\n",
> > > + __func__, cmdnr);
> > > + ret = -EFAULT;
> > > + }
> > > + break;
> > > + }
> > > +
> > > + case VC_SM_CMA_CMD_IMPORT_DMABUF:
> > > + {
> > > + struct vc_sm_cma_ioctl_import_dmabuf ioparam;
> > > + struct dma_buf *new_dmabuf;
> > > +
> > > + /* Get the parameter data. */
> > > + if (copy_from_user
> > > + (&ioparam, (void *)arg, sizeof(ioparam)) != 0) {
> > > + pr_err("[%s]: failed to copy-from-user for cmd
> > > %x\n",
> > > + __func__, cmdnr);
> > > + ret = -EFAULT;
> > > + break;
> > > + }
> > > +
> > > + ret = vc_sm_cma_import_dmabuf_internal(file_data,
> > > + NULL,
> > > + ioparam.dmabuf_f
> > > d,
> > > + &new_dmabuf);
> > > +
> > > + if (!ret) {
> > > + struct vc_sm_buffer *buf = new_dmabuf->priv;
> > > +
> > > + ioparam.size = buf->size;
> > > + ioparam.handle = dma_buf_fd(new_dmabuf,
> > > + O_CLOEXEC);
> > > + ioparam.vc_handle = buf->vc_handle;
> > > + ioparam.dma_addr = buf->dma_addr;
> > > +
> > > + if (ioparam.handle < 0 ||
> > > + (copy_to_user((void *)arg, &ioparam,
> > > + sizeof(ioparam)) != 0)) {
> > > + dma_buf_put(new_dmabuf);
> > > + /* FIXME: Release allocation */
> > > + ret = -EFAULT;
> > > + }
> > > + }
> > > + break;
> > > + }
> > > +
> > > +#ifndef CONFIG_ARM64
> > > + /*
> > > + * Flush/Invalidate the cache for a given mapping.
> > > + * Blocks must be pinned (i.e. accessed) before this call.
> > > + */
> > > + case VC_SM_CMA_CMD_CLEAN_INVALID2:
> > > + ret = vc_sm_cma_clean_invalid2(cmdnr, arg);
> > > + break;
> > > +#endif
> > > +
> > > + default:
> > > + pr_debug("[%s]: cmd %x tgid %u, owner %u\n", __func__,
> > > cmdnr,
> > > + current->tgid, file_data->pid);
> > > +
> > > + ret = -EINVAL;
> > > + break;
> > > + }
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +#ifdef CONFIG_COMPAT
> > > +struct vc_sm_cma_ioctl_clean_invalid2_32 {
> > > + u32 op_count;
> > > + struct vc_sm_cma_ioctl_clean_invalid_block_32 {
> > > + u16 invalidate_mode;
> > > + u16 block_count;
> > > + compat_uptr_t start_address;
> > > + u32 block_size;
> > > + u32 inter_block_stride;
> > > + } s[0];
> > > +};
> > > +
> > > +#define VC_SM_CMA_CMD_CLEAN_INVALID2_32\
> > > + _IOR(VC_SM_CMA_MAGIC_TYPE, VC_SM_CMA_CMD_CLEAN_INVALID2,\
> > > + struct vc_sm_cma_ioctl_clean_invalid2_32)
> > > +
> > > +static long vc_sm_cma_compat_ioctl(struct file *file, unsigned int
> > > cmd,
> > > + unsigned long arg)
> > > +{
> > > + switch (cmd) {
> > > + case VC_SM_CMA_CMD_CLEAN_INVALID2_32:
> > > + /* FIXME */
> > > + return -EINVAL;
> > > +
> > > + default:
> > > + return vc_sm_cma_ioctl(file, cmd, arg);
> > > + }
> > > +}
> > > +#endif
> > > +
> > > +/* Device operations that we managed in this driver. */
> > > +static const struct file_operations vc_sm_ops = {
> > > + .owner = THIS_MODULE,
> > > + .unlocked_ioctl = vc_sm_cma_ioctl,
> > > +#ifdef CONFIG_COMPAT
> > > + .compat_ioctl = vc_sm_cma_compat_ioctl,
> > > +#endif
> > > + .open = vc_sm_cma_open,
> > > + .release = vc_sm_cma_release,
> > > +};
> > > +
> > > +/* Driver load/unload functions */
> > > +/* Videocore connected. */
> > > +static void vc_sm_connected_init(void)
> > > +{
> > > + int ret;
> > > + struct vchi_instance_handle *vchi_instance;
> > > + struct vc_sm_version version;
> > > + struct vc_sm_result_t version_result;
> > > +
> > > + pr_info("[%s]: start\n", __func__);
> > > +
> > > + /*
> > > + * Initialize and create a VCHI connection for the shared
> > > memory service
> > > + * running on videocore.
> > > + */
> > > + ret = vchi_initialise(&vchi_instance);
> > > + if (ret) {
> > > + pr_err("[%s]: failed to initialise VCHI instance
> > > (ret=%d)\n",
> > > + __func__, ret);
> > > +
> > > + return;
> > > + }
> > > +
> > > + ret = vchi_connect(vchi_instance);
> > > + if (ret) {
> > > + pr_err("[%s]: failed to connect VCHI instance
> > > (ret=%d)\n",
> > > + __func__, ret);
> > > +
> > > + return;
> > > + }
> > > +
> > > + /* Initialize an instance of the shared memory service. */
> > > + sm_state->sm_handle = vc_sm_cma_vchi_init(vchi_instance, 1,
> > > + vc_sm_vpu_event);
> > > + if (!sm_state->sm_handle) {
> > > + pr_err("[%s]: failed to initialize shared memory
> > > service\n",
> > > + __func__);
> > > +
> > > + return;
> > > + }
> > > +
> > > + /* Create a debug fs directory entry (root). */
> > > + sm_state->dir_root = debugfs_create_dir(VC_SM_DIR_ROOT_NAME,
> > > NULL);
> > > +
> > > + sm_state->dir_state.show = &vc_sm_cma_global_state_show;
> > > + sm_state->dir_state.dir_entry =
> > > + debugfs_create_file(VC_SM_STATE, 0444, sm_state-
> > > >dir_root,
> > > + &sm_state->dir_state,
> > > + &vc_sm_cma_debug_fs_fops);
> > > +
> > > + INIT_LIST_HEAD(&sm_state->buffer_list);
> > > +
> > > + /* Create a shared memory device. */
> > > + sm_state->misc_dev.minor = MISC_DYNAMIC_MINOR;
> > > + sm_state->misc_dev.name = DEVICE_NAME;
> > > + sm_state->misc_dev.fops = &vc_sm_ops;
> > > + sm_state->misc_dev.parent = NULL;
> > > + /* Temporarily set as 666 until udev rules have been sorted */
> > > + sm_state->misc_dev.mode = 0666;
> > > + ret = misc_register(&sm_state->misc_dev);
> > > + if (ret) {
> > > + pr_err("vcsm-cma: failed to register misc device.\n");
> > > + goto err_remove_debugfs;
> > > + }
> > > +
> > > + sm_state->data_knl = vc_sm_cma_create_priv_data(0);
> > > + if (!sm_state->data_knl) {
> > > + pr_err("[%s]: failed to create kernel private data
> > > tracker\n",
> > > + __func__);
> > > + goto err_remove_misc_dev;
> > > + }
> > > +
> > > + version.version = 2;
> > > + ret = vc_sm_cma_vchi_client_version(sm_state->sm_handle,
> > > &version,
> > > + &version_result,
> > > + &sm_state->int_trans_id);
> > > + if (ret) {
> > > + pr_err("[%s]: Failed to send version request %d\n",
> > > __func__,
> > > + ret);
> > > + }
> > > +
> > > + /* Done! */
> > > + sm_inited = 1;
> > > + pr_info("[%s]: installed successfully\n", __func__);
> > > + return;
> > > +
> > > +err_remove_misc_dev:
> > > + misc_deregister(&sm_state->misc_dev);
> > > +err_remove_debugfs:
> > > + debugfs_remove_recursive(sm_state->dir_root);
> > > + vc_sm_cma_vchi_stop(&sm_state->sm_handle);
> > > +}
> > > +
> > > +/* Driver loading. */
> > > +static int bcm2835_vc_sm_cma_probe(struct platform_device *pdev)
> > > +{
> > > + pr_info("%s: Videocore shared memory driver\n", __func__);
> > > +
> > > + sm_state = devm_kzalloc(&pdev->dev, sizeof(*sm_state),
> > > GFP_KERNEL);
> > > + if (!sm_state)
> > > + return -ENOMEM;
> > > + sm_state->pdev = pdev;
> > > + mutex_init(&sm_state->map_lock);
> > > +
> > > + spin_lock_init(&sm_state->kernelid_map_lock);
> > > + idr_init_base(&sm_state->kernelid_map, 1);
> > > +
> > > + pdev->dev.dma_parms = devm_kzalloc(&pdev->dev,
> > > + sizeof(*pdev-
> > > >dev.dma_parms),
> > > + GFP_KERNEL);
> > > + /* dma_set_max_seg_size checks if dma_parms is NULL. */
> > > + dma_set_max_seg_size(&pdev->dev, 0x3FFFFFFF);
> > > +
> > > + vchiq_add_connected_callback(vc_sm_connected_init);
> > > + return 0;
> > > +}
> > > +
> > > +/* Driver unloading. */
> > > +static int bcm2835_vc_sm_cma_remove(struct platform_device *pdev)
> > > +{
> > > + pr_debug("[%s]: start\n", __func__);
> > > + if (sm_inited) {
> > > + misc_deregister(&sm_state->misc_dev);
> > > +
> > > + /* Remove all proc entries. */
> > > + debugfs_remove_recursive(sm_state->dir_root);
> > > +
> > > + /* Stop the videocore shared memory service. */
> > > + vc_sm_cma_vchi_stop(&sm_state->sm_handle);
> > > + }
> > > +
> > > + if (sm_state) {
> > > + idr_destroy(&sm_state->kernelid_map);
> > > +
> > > + /* Free the memory for the state structure. */
> > > + mutex_destroy(&sm_state->map_lock);
> > > + }
> > > +
> > > + pr_debug("[%s]: end\n", __func__);
> > > + return 0;
> > > +}
> > > +
> > > +/* Kernel API calls */
> > > +/* Get an internal resource handle mapped from the external one. */
> > > +int vc_sm_cma_int_handle(void *handle)
> > > +{
> > > + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> > > + struct vc_sm_buffer *buf;
> > > +
> > > + /* Validate we can work with this device. */
> > > + if (!sm_state || !handle) {
> > > + pr_err("[%s]: invalid input\n", __func__);
> > > + return 0;
> > > + }
> > > +
> > > + buf = (struct vc_sm_buffer *)dma_buf->priv;
> > > + return buf->vc_handle;
> > > +}
> > > +EXPORT_SYMBOL_GPL(vc_sm_cma_int_handle);
> > > +
> > > +/* Free a previously allocated shared memory handle and block. */
> > > +int vc_sm_cma_free(void *handle)
> > > +{
> > > + struct dma_buf *dma_buf = (struct dma_buf *)handle;
> > > +
> > > + /* Validate we can work with this device. */
> > > + if (!sm_state || !handle) {
> > > + pr_err("[%s]: invalid input\n", __func__);
> > > + return -EPERM;
> > > + }
> > > +
> > > + pr_debug("%s: handle %p/dmabuf %p\n", __func__, handle,
> > > dma_buf);
> > > +
> > > + dma_buf_put(dma_buf);
> > > +
> > > + return 0;
> > > +}
> > > +EXPORT_SYMBOL_GPL(vc_sm_cma_free);
> > > +
> > > +/* Import a dmabuf to be shared with VC. */
> > > +int vc_sm_cma_import_dmabuf(struct dma_buf *src_dmabuf, void
> > > **handle)
> > > +{
> > > + struct dma_buf *new_dma_buf;
> > > + struct vc_sm_buffer *buf;
> > > + int ret;
> > > +
> > > + /* Validate we can work with this device. */
> > > + if (!sm_state || !src_dmabuf || !handle) {
> > > + pr_err("[%s]: invalid input\n", __func__);
> > > + return -EPERM;
> > > + }
> > > +
> > > + ret = vc_sm_cma_import_dmabuf_internal(sm_state->data_knl,
> > > src_dmabuf,
> > > + -1, &new_dma_buf);
> > > +
> > > + if (!ret) {
> > > + pr_debug("%s: imported to ptr %p\n", __func__,
> > > new_dma_buf);
> > > + buf = (struct vc_sm_buffer *)new_dma_buf->priv;
> > > +
> > > + /* Assign valid handle at this time.*/
> > > + *handle = new_dma_buf;
> > > + } else {
> > > + /*
> > > + * succeeded in importing the dma_buf, but then
> > > + * failed to look it up again. How?
> > > + * Release the fd again.
> > > + */
> > > + pr_err("%s: imported vc_sm_cma_get_buffer failed %d\n",
> > > + __func__, ret);
> > > + }
> > > +
> > > + return ret;
> > > +}
> > > +EXPORT_SYMBOL_GPL(vc_sm_cma_import_dmabuf);
> > > +
> > > +static struct platform_driver bcm2835_vcsm_cma_driver = {
> > > + .probe = bcm2835_vc_sm_cma_probe,
> > > + .remove = bcm2835_vc_sm_cma_remove,
> > > + .driver = {
> > > + .name = DEVICE_NAME,
> > > + .owner = THIS_MODULE,
> > > + },
> > > +};
> > > +
> > > +module_platform_driver(bcm2835_vcsm_cma_driver);
> > > +
> > > +MODULE_AUTHOR("Dave Stevenson");
> > > +MODULE_DESCRIPTION("VideoCore CMA Shared Memory Driver");
> > > +MODULE_LICENSE("GPL v2");
> > > +MODULE_ALIAS("platform:vcsm-cma");
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > > new file mode 100644
> > > index 000000000000..f1c7b95b14ce
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> > > @@ -0,0 +1,84 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +/*
> > > + * VideoCore Shared Memory driver using CMA.
> > > + *
> > > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > > + *
> > > + */
> > > +
> > > +#ifndef VC_SM_H
> > > +#define VC_SM_H
> > > +
> > > +#include <linux/device.h>
> > > +#include <linux/dma-direction.h>
> > > +#include <linux/kref.h>
> > > +#include <linux/mm_types.h>
> > > +#include <linux/mutex.h>
> > > +#include <linux/rbtree.h>
> > > +#include <linux/sched.h>
> > > +#include <linux/shrinker.h>
> > > +#include <linux/types.h>
> > > +#include <linux/miscdevice.h>
> > > +
> > > +#define VC_SM_MAX_NAME_LEN 32
> > > +
> > > +enum vc_sm_vpu_mapping_state {
> > > + VPU_NOT_MAPPED,
> > > + VPU_MAPPED,
> > > + VPU_UNMAPPING
> > > +};
> > > +
> > > +struct vc_sm_alloc_data {
> > > + unsigned long num_pages;
> > > + void *priv_virt;
> > > + struct sg_table *sg_table;
> > > +};
> > > +
> > > +struct vc_sm_imported {
> > > + struct dma_buf *dma_buf;
> > > + struct dma_buf_attachment *attach;
> > > + struct sg_table *sgt;
> > > +};
> > > +
> > > +struct vc_sm_buffer {
> > > + struct list_head global_buffer_list; /* Global list of
> > > buffers. */
> > > +
> > > + /* Index in the kernel_id idr so that we can find the
> > > + * mmal_msg_context again when servicing the VCHI reply.
> > > + */
> > > + int kernel_id;
> > > +
> > > + size_t size;
> > > +
> > > + /* Lock over all the following state for this buffer */
> > > + struct mutex lock;
> > > + struct list_head attachments;
> > > +
> > > + char name[VC_SM_MAX_NAME_LEN];
> > > +
> > > + int in_use:1; /* Kernel is still using this resource */
> > > + int imported:1; /* Imported dmabuf */
> > > +
> > > + enum vc_sm_vpu_mapping_state vpu_state;
> > > + u32 vc_handle; /* VideoCore handle for this buffer */
> > > + int vpu_allocated; /*
> > > + * The VPU made this allocation.
> > > Release the
> > > + * local dma_buf when the VPU releases
> > > the
> > > + * resource.
> > > + */
> > > +
> > > + /* DMABUF related fields */
> > > + struct dma_buf *dma_buf;
> > > + dma_addr_t dma_addr;
> > > + void *cookie;
> > > +
> > > + struct vc_sm_privdata_t *private;
> > > +
> > > + union {
> > > + struct vc_sm_alloc_data alloc;
> > > + struct vc_sm_imported import;
> > > + };
> > > +};
> > > +
> > > +#endif
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> > > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> > > new file mode 100644
> > > index 000000000000..6a203c60bf7f
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> > > @@ -0,0 +1,505 @@
> > > +// SPDX-License-Identifier: GPL-2.0
> > > +/*
> > > + * VideoCore Shared Memory CMA allocator
> > > + *
> > > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > > + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> > > + *
> > > + * Based on vmcs_sm driver from Broadcom Corporation.
> > > + *
> > > + */
> > > +
> > > +/* ---- Include Files ----------------------------------------------
> > > ------- */
> > > +#include <linux/completion.h>
> > > +#include <linux/kernel.h>
> > > +#include <linux/kthread.h>
> > > +#include <linux/list.h>
> > > +#include <linux/mutex.h>
> > > +#include <linux/semaphore.h>
> > > +#include <linux/slab.h>
> > > +#include <linux/types.h>
> > > +
> > > +#include "vc_sm_cma_vchi.h"
> > > +
> > > +#define VC_SM_VER 1
> > > +#define VC_SM_MIN_VER 0
> > > +
> > > +/* ---- Private Constants and Types --------------------------------
> > > ------ */
> > > +
> > > +/* Command blocks come from a pool */
> > > +#define SM_MAX_NUM_CMD_RSP_BLKS 32
> > > +
> > > +struct sm_cmd_rsp_blk {
> > > + struct list_head head; /* To create lists */
> > > + /* To be signaled when the response is there */
> > > + struct completion cmplt;
> > > +
> > > + u32 id;
> > > + u16 length;
> > > +
> > > + u8 msg[VC_SM_MAX_MSG_LEN];
> > > +
> > > + uint32_t wait:1;
> > > + uint32_t sent:1;
> > > + uint32_t alloc:1;
> > > +
> > > +};
> > > +
> > > +struct sm_instance {
> > > + u32 num_connections;
> > > + struct vchi_service_handle
> > > *vchi_handle[VCHI_MAX_NUM_CONNECTIONS];
> > > + struct task_struct *io_thread;
> > > + struct completion io_cmplt;
> > > +
> > > + vpu_event_cb vpu_event;
> > > +
> > > + /* Mutex over the following lists */
> > > + struct mutex lock;
> > > + u32 trans_id;
> > > + struct list_head cmd_list;
> > > + struct list_head rsp_list;
> > > + struct list_head dead_list;
> > > +
> > > + struct sm_cmd_rsp_blk free_blk[SM_MAX_NUM_CMD_RSP_BLKS];
> > > +
> > > + /* Mutex over the free_list */
> > > + struct mutex free_lock;
> > > + struct list_head free_list;
> > > +
> > > + struct semaphore free_sema;
> > > +
> > > +};
> > > +
> > > +/* ---- Private Variables ------------------------------------------
> > > ------ */
> > > +
> > > +/* ---- Private Function Prototypes --------------------------------
> > > ------ */
> > > +
> > > +/* ---- Private Functions ------------------------------------------
> > > ------ */
> > > +static int
> > > +bcm2835_vchi_msg_queue(struct vchi_service_handle *handle,
> > > + void *data,
> > > + unsigned int size)
> > > +{
> > > + return vchi_queue_kernel_message(handle,
> > > + data,
> > > + size);
> > > +}
> > > +
> > > +static struct
> > > +sm_cmd_rsp_blk *vc_vchi_cmd_create(struct sm_instance *instance,
> > > + enum vc_sm_msg_type id, void *msg,
> > > + u32 size, int wait)
> > > +{
> > > + struct sm_cmd_rsp_blk *blk;
> > > + struct vc_sm_msg_hdr_t *hdr;
> > > +
> > > + if (down_interruptible(&instance->free_sema)) {
> > > + blk = kmalloc(sizeof(*blk), GFP_KERNEL);
> > > + if (!blk)
> > > + return NULL;
> > > +
> > > + blk->alloc = 1;
> > > + init_completion(&blk->cmplt);
> > > + } else {
> > > + mutex_lock(&instance->free_lock);
> > > + blk =
> > > + list_first_entry(&instance->free_list,
> > > + struct sm_cmd_rsp_blk, head);
> > > + list_del(&blk->head);
> > > + mutex_unlock(&instance->free_lock);
> > > + }
> > > +
> > > + blk->sent = 0;
> > > + blk->wait = wait;
> > > + blk->length = sizeof(*hdr) + size;
> > > +
> > > + hdr = (struct vc_sm_msg_hdr_t *)blk->msg;
> > > + hdr->type = id;
> > > + mutex_lock(&instance->lock);
> > > + instance->trans_id++;
> > > + /*
> > > + * Retain the top bit for identifying asynchronous events, or
> > > VPU cmds.
> > > + */
> > > + instance->trans_id &= ~0x80000000;
> > > + hdr->trans_id = instance->trans_id;
> > > + blk->id = instance->trans_id;
> > > + mutex_unlock(&instance->lock);
> > > +
> > > + if (size)
> > > + memcpy(hdr->body, msg, size);
> > > +
> > > + return blk;
> > > +}
> > > +
> > > +static void
> > > +vc_vchi_cmd_delete(struct sm_instance *instance, struct
> > > sm_cmd_rsp_blk *blk)
> > > +{
> > > + if (blk->alloc) {
> > > + kfree(blk);
> > > + return;
> > > + }
> > > +
> > > + mutex_lock(&instance->free_lock);
> > > + list_add(&blk->head, &instance->free_list);
> > > + mutex_unlock(&instance->free_lock);
> > > + up(&instance->free_sema);
> > > +}
> > > +
> > > +static void vc_sm_cma_vchi_rx_ack(struct sm_instance *instance,
> > > + struct sm_cmd_rsp_blk *cmd,
> > > + struct vc_sm_result_t *reply,
> > > + u32 reply_len)
> > > +{
> > > + mutex_lock(&instance->lock);
> > > + list_for_each_entry(cmd,
> > > + &instance->rsp_list,
> > > + head) {
> > > + if (cmd->id == reply->trans_id)
> > > + break;
> > > + }
> > > + mutex_unlock(&instance->lock);
> > > +
> > > + if (&cmd->head == &instance->rsp_list) {
> > > + //pr_debug("%s: received response %u, throw away...",
> > > + pr_err("%s: received response %u, throw away...",
> > > + __func__,
> > > + reply->trans_id);
> > > + } else if (reply_len > sizeof(cmd->msg)) {
> > > + pr_err("%s: reply too big (%u) %u, throw away...",
> > > + __func__, reply_len,
> > > + reply->trans_id);
> > > + } else {
> > > + memcpy(cmd->msg, reply,
> > > + reply_len);
> > > + complete(&cmd->cmplt);
> > > + }
> > > +}
> > > +
> > > +static int vc_sm_cma_vchi_videocore_io(void *arg)
> > > +{
> > > + struct sm_instance *instance = arg;
> > > + struct sm_cmd_rsp_blk *cmd = NULL, *cmd_tmp;
> > > + struct vc_sm_result_t *reply;
> > > + u32 reply_len;
> > > + s32 status;
> > > + int svc_use = 1;
> > > +
> > > + while (1) {
> > > + if (svc_use)
> > > + vchi_service_release(instance->vchi_handle[0]);
> > > + svc_use = 0;
> > > +
> > > + if (wait_for_completion_interruptible(&instance-
> > > >io_cmplt))
> > > + continue;
> > > +
> > > + vchi_service_use(instance->vchi_handle[0]);
> > > + svc_use = 1;
> > > +
> > > + do {
> > > + /*
> > > + * Get new command and move it to response list
> > > + */
> > > + mutex_lock(&instance->lock);
> > > + if (list_empty(&instance->cmd_list)) {
> > > + /* no more commands to process */
> > > + mutex_unlock(&instance->lock);
> > > + break;
> > > + }
> > > + cmd = list_first_entry(&instance->cmd_list,
> > > + struct sm_cmd_rsp_blk,
> > > head);
> > > + list_move(&cmd->head, &instance->rsp_list);
> > > + cmd->sent = 1;
> > > + mutex_unlock(&instance->lock);
> > > +
> > > + /* Send the command */
> > > + status =
> > > + bcm2835_vchi_msg_queue(instance-
> > > >vchi_handle[0],
> > > + cmd->msg, cmd-
> > > >length);
> > > + if (status) {
> > > + pr_err("%s: failed to queue message
> > > (%d)",
> > > + __func__, status);
> > > + }
> > > +
> > > + /* If no reply is needed then we're done */
> > > + if (!cmd->wait) {
> > > + mutex_lock(&instance->lock);
> > > + list_del(&cmd->head);
> > > + mutex_unlock(&instance->lock);
> > > + vc_vchi_cmd_delete(instance, cmd);
> > > + continue;
> > > + }
> > > +
> > > + if (status) {
> > > + complete(&cmd->cmplt);
> > > + continue;
> > > + }
> > > +
> > > + } while (1);
> > > +
> > > + while (!vchi_msg_peek(instance->vchi_handle[0], (void
> > > **)&reply,
> > > + &reply_len, VCHI_FLAGS_NONE)) {
> > > + if (reply->trans_id & 0x80000000) {
> > > + /* Async event or cmd from the VPU */
> > > + if (instance->vpu_event)
> > > + instance->vpu_event(instance,
> > > reply,
> > > + reply_len);
> > > + } else {
> > > + vc_sm_cma_vchi_rx_ack(instance, cmd,
> > > reply,
> > > + reply_len);
> > > + }
> > > +
> > > + vchi_msg_remove(instance->vchi_handle[0]);
> > > + }
> > > +
> > > + /* Go through the dead list and free them */
> > > + mutex_lock(&instance->lock);
> > > + list_for_each_entry_safe(cmd, cmd_tmp, &instance-
> > > >dead_list,
> > > + head) {
> > > + list_del(&cmd->head);
> > > + vc_vchi_cmd_delete(instance, cmd);
> > > + }
> > > + mutex_unlock(&instance->lock);
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +static void vc_sm_cma_vchi_callback(void *param,
> > > + const enum vchi_callback_reason
> > > reason,
> > > + void *msg_handle)
> > > +{
> > > + struct sm_instance *instance = param;
> > > +
> > > + (void)msg_handle;
> > > +
> > > + switch (reason) {
> > > + case VCHI_CALLBACK_MSG_AVAILABLE:
> > > + complete(&instance->io_cmplt);
> > > + break;
> > > +
> > > + case VCHI_CALLBACK_SERVICE_CLOSED:
> > > + pr_info("%s: service CLOSED!!", __func__);
> > > + default:
> > > + break;
> > > + }
> > > +}
> > > +
> > > +struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle
> > > *vchi_instance,
> > > + unsigned int num_connections,
> > > + vpu_event_cb vpu_event)
> > > +{
> > > + u32 i;
> > > + struct sm_instance *instance;
> > > + int status;
> > > +
> > > + pr_debug("%s: start", __func__);
> > > +
> > > + if (num_connections > VCHI_MAX_NUM_CONNECTIONS) {
> > > + pr_err("%s: unsupported number of connections %u
> > > (max=%u)",
> > > + __func__, num_connections,
> > > VCHI_MAX_NUM_CONNECTIONS);
> > > +
> > > + goto err_null;
> > > + }
> > > + /* Allocate memory for this instance */
> > > + instance = kzalloc(sizeof(*instance), GFP_KERNEL);
> > > +
> > > + /* Misc initialisations */
> > > + mutex_init(&instance->lock);
> > > + init_completion(&instance->io_cmplt);
> > > + INIT_LIST_HEAD(&instance->cmd_list);
> > > + INIT_LIST_HEAD(&instance->rsp_list);
> > > + INIT_LIST_HEAD(&instance->dead_list);
> > > + INIT_LIST_HEAD(&instance->free_list);
> > > + sema_init(&instance->free_sema, SM_MAX_NUM_CMD_RSP_BLKS);
> > > + mutex_init(&instance->free_lock);
> > > + for (i = 0; i < SM_MAX_NUM_CMD_RSP_BLKS; i++) {
> > > + init_completion(&instance->free_blk[i].cmplt);
> > > + list_add(&instance->free_blk[i].head, &instance-
> > > >free_list);
> > > + }
> > > +
> > > + /* Open the VCHI service connections */
> > > + instance->num_connections = num_connections;
> > > + for (i = 0; i < num_connections; i++) {
> > > + struct service_creation params = {
> > > + .version = VCHI_VERSION_EX(VC_SM_VER,
> > > VC_SM_MIN_VER),
> > > + .service_id = VC_SM_SERVER_NAME,
> > > + .callback = vc_sm_cma_vchi_callback,
> > > + .callback_param = instance,
> > > + };
> > > +
> > > + status = vchi_service_open(vchi_instance,
> > > + ¶ms, &instance-
> > > >vchi_handle[i]);
> > > + if (status) {
> > > + pr_err("%s: failed to open VCHI service (%d)",
> > > + __func__, status);
> > > +
> > > + goto err_close_services;
> > > + }
> > > + }
> > > +
> > > + /* Create the thread which takes care of all io to/from
> > > videoocore. */
> > > + instance->io_thread =
> > > kthread_create(&vc_sm_cma_vchi_videocore_io,
> > > + (void *)instance, "SMIO");
> > > + if (!instance->io_thread) {
> > > + pr_err("%s: failed to create SMIO thread", __func__);
> > > +
> > > + goto err_close_services;
> > > + }
> > > + instance->vpu_event = vpu_event;
> > > + set_user_nice(instance->io_thread, -10);
> > > + wake_up_process(instance->io_thread);
> > > +
> > > + pr_debug("%s: success - instance %p", __func__, instance);
> > > + return instance;
> > > +
> > > +err_close_services:
> > > + for (i = 0; i < instance->num_connections; i++) {
> > > + if (instance->vchi_handle[i])
> > > + vchi_service_close(instance->vchi_handle[i]);
> > > + }
> > > + kfree(instance);
> > > +err_null:
> > > + pr_debug("%s: FAILED", __func__);
> > > + return NULL;
> > > +}
> > > +
> > > +int vc_sm_cma_vchi_stop(struct sm_instance **handle)
> > > +{
> > > + struct sm_instance *instance;
> > > + u32 i;
> > > +
> > > + if (!handle) {
> > > + pr_err("%s: invalid pointer to handle %p", __func__,
> > > handle);
> > > + goto lock;
> > > + }
> > > +
> > > + if (!*handle) {
> > > + pr_err("%s: invalid handle %p", __func__, *handle);
> > > + goto lock;
> > > + }
> > > +
> > > + instance = *handle;
> > > +
> > > + /* Close all VCHI service connections */
> > > + for (i = 0; i < instance->num_connections; i++) {
> > > + s32 success;
> > > +
> > > + vchi_service_use(instance->vchi_handle[i]);
> > > +
> > > + success = vchi_service_close(instance->vchi_handle[i]);
> > > + }
> > > +
> > > + kfree(instance);
> > > +
> > > + *handle = NULL;
> > > + return 0;
> > > +
> > > +lock:
> > > + return -EINVAL;
> > > +}
> > > +
> > > +static int vc_sm_cma_vchi_send_msg(struct sm_instance *handle,
> > > + enum vc_sm_msg_type msg_id, void
> > > *msg,
> > > + u32 msg_size, void *result, u32
> > > result_size,
> > > + u32 *cur_trans_id, u8 wait_reply)
> > > +{
> > > + int status = 0;
> > > + struct sm_instance *instance = handle;
> > > + struct sm_cmd_rsp_blk *cmd_blk;
> > > +
> > > + if (!handle) {
> > > + pr_err("%s: invalid handle", __func__);
> > > + return -EINVAL;
> > > + }
> > > + if (!msg) {
> > > + pr_err("%s: invalid msg pointer", __func__);
> > > + return -EINVAL;
> > > + }
> > > +
> > > + cmd_blk =
> > > + vc_vchi_cmd_create(instance, msg_id, msg, msg_size,
> > > wait_reply);
> > > + if (!cmd_blk) {
> > > + pr_err("[%s]: failed to allocate global tracking
> > > resource",
> > > + __func__);
> > > + return -ENOMEM;
> > > + }
> > > +
> > > + if (cur_trans_id)
> > > + *cur_trans_id = cmd_blk->id;
> > > +
> > > + mutex_lock(&instance->lock);
> > > + list_add_tail(&cmd_blk->head, &instance->cmd_list);
> > > + mutex_unlock(&instance->lock);
> > > + complete(&instance->io_cmplt);
> > > +
> > > + if (!wait_reply)
> > > + /* We're done */
> > > + return 0;
> > > +
> > > + /* Wait for the response */
> > > + if (wait_for_completion_interruptible(&cmd_blk->cmplt)) {
> > > + mutex_lock(&instance->lock);
> > > + if (!cmd_blk->sent) {
> > > + list_del(&cmd_blk->head);
> > > + mutex_unlock(&instance->lock);
> > > + vc_vchi_cmd_delete(instance, cmd_blk);
> > > + return -ENXIO;
> > > + }
> > > +
> > > + list_move(&cmd_blk->head, &instance->dead_list);
> > > + mutex_unlock(&instance->lock);
> > > + complete(&instance->io_cmplt);
> > > + return -EINTR; /* We're done */
> > > + }
> > > +
> > > + if (result && result_size) {
> > > + memcpy(result, cmd_blk->msg, result_size);
> > > + } else {
> > > + struct vc_sm_result_t *res =
> > > + (struct vc_sm_result_t *)cmd_blk->msg;
> > > + status = (res->success == 0) ? 0 : -ENXIO;
> > > + }
> > > +
> > > + mutex_lock(&instance->lock);
> > > + list_del(&cmd_blk->head);
> > > + mutex_unlock(&instance->lock);
> > > + vc_vchi_cmd_delete(instance, cmd_blk);
> > > + return status;
> > > +}
> > > +
> > > +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct
> > > vc_sm_free_t *msg,
> > > + u32 *cur_trans_id)
> > > +{
> > > + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_FREE,
> > > + msg, sizeof(*msg), 0, 0,
> > > cur_trans_id, 0);
> > > +}
> > > +
> > > +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct
> > > vc_sm_import *msg,
> > > + struct vc_sm_import_result *result, u32
> > > *cur_trans_id)
> > > +{
> > > + return vc_sm_cma_vchi_send_msg(handle, VC_SM_MSG_TYPE_IMPORT,
> > > + msg, sizeof(*msg), result,
> > > sizeof(*result),
> > > + cur_trans_id, 1);
> > > +}
> > > +
> > > +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> > > + struct vc_sm_version *msg,
> > > + struct vc_sm_result_t *result,
> > > + u32 *cur_trans_id)
> > > +{
> > > + return vc_sm_cma_vchi_send_msg(handle,
> > > VC_SM_MSG_TYPE_CLIENT_VERSION,
> > > + //msg, sizeof(*msg), result,
> > > sizeof(*result),
> > > + //cur_trans_id, 1);
> > > + msg, sizeof(*msg), NULL, 0,
> > > + cur_trans_id, 0);
> > > +}
> > > +
> > > +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> > > + struct
> > > vc_sm_vc_mem_request_result *msg,
> > > + uint32_t *cur_trans_id)
> > > +{
> > > + return vc_sm_cma_vchi_send_msg(handle,
> > > + VC_SM_MSG_TYPE_VC_MEM_REQUEST_RE
> > > PLY,
> > > + msg, sizeof(*msg), 0, 0,
> > > cur_trans_id,
> > > + 0);
> > > +}
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> > > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> > > new file mode 100644
> > > index 000000000000..e8db34bd1e91
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> > > @@ -0,0 +1,63 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +/*
> > > + * VideoCore Shared Memory CMA allocator
> > > + *
> > > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > > + * Copyright 2011-2012 Broadcom Corporation. All rights reserved.
> > > + *
> > > + * Based on vmcs_sm driver from Broadcom Corporation.
> > > + *
> > > + */
> > > +
> > > +#ifndef __VC_SM_CMA_VCHI_H__INCLUDED__
> > > +#define __VC_SM_CMA_VCHI_H__INCLUDED__
> > > +
> > > +#include "interface/vchi/vchi.h"
> > > +
> > > +#include "vc_sm_defs.h"
> > > +
> > > +/*
> > > + * Forward declare.
> > > + */
> > > +struct sm_instance;
> > > +
> > > +typedef void (*vpu_event_cb)(struct sm_instance *instance,
> > > + struct vc_sm_result_t *reply, int
> > > reply_len);
> > > +
> > > +/*
> > > + * Initialize the shared memory service, opens up vchi connection to
> > > talk to it.
> > > + */
> > > +struct sm_instance *vc_sm_cma_vchi_init(struct vchi_instance_handle
> > > *vchi_instance,
> > > + unsigned int num_connections,
> > > + vpu_event_cb vpu_event);
> > > +
> > > +/*
> > > + * Terminates the shared memory service.
> > > + */
> > > +int vc_sm_cma_vchi_stop(struct sm_instance **handle);
> > > +
> > > +/*
> > > + * Ask the shared memory service to free up some memory that was
> > > previously
> > > + * allocated by the vc_sm_cma_vchi_alloc function call.
> > > + */
> > > +int vc_sm_cma_vchi_free(struct sm_instance *handle, struct
> > > vc_sm_free_t *msg,
> > > + u32 *cur_trans_id);
> > > +
> > > +/*
> > > + * Import a contiguous block of memory and wrap it in a GPU
> > > MEM_HANDLE_T.
> > > + */
> > > +int vc_sm_cma_vchi_import(struct sm_instance *handle, struct
> > > vc_sm_import *msg,
> > > + struct vc_sm_import_result *result,
> > > + u32 *cur_trans_id);
> > > +
> > > +int vc_sm_cma_vchi_client_version(struct sm_instance *handle,
> > > + struct vc_sm_version *msg,
> > > + struct vc_sm_result_t *result,
> > > + u32 *cur_trans_id);
> > > +
> > > +int vc_sm_vchi_client_vc_mem_req_reply(struct sm_instance *handle,
> > > + struct
> > > vc_sm_vc_mem_request_result *msg,
> > > + uint32_t *cur_trans_id);
> > > +
> > > +#endif /* __VC_SM_CMA_VCHI_H__INCLUDED__ */
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> > > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> > > new file mode 100644
> > > index 000000000000..8a0d1f6dbfe8
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> > > @@ -0,0 +1,300 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +/*
> > > + * VideoCore Shared Memory CMA allocator
> > > + *
> > > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > > + *
> > > + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom
> > > Corporation.
> > > + * All IPC messages are copied across to this file, even if the vc-
> > > sm-cma
> > > + * driver is not currently using them.
> > > + *
> > > +
> > > *********************************************************************
> > > *******
> > > + */
> > > +
> > > +#ifndef __VC_SM_DEFS_H__INCLUDED__
> > > +#define __VC_SM_DEFS_H__INCLUDED__
> > > +
> > > +/* FourCC code used for VCHI connection */
> > > +#define VC_SM_SERVER_NAME MAKE_FOURCC("SMEM")
> > > +
> > > +/* Maximum message length */
> > > +#define VC_SM_MAX_MSG_LEN (sizeof(union vc_sm_msg_union_t) + \
> > > + sizeof(struct vc_sm_msg_hdr_t))
> > > +#define VC_SM_MAX_RSP_LEN (sizeof(union vc_sm_msg_union_t))
> > > +
> > > +/* Resource name maximum size */
> > > +#define VC_SM_RESOURCE_NAME 32
> > > +
> > > +/*
> > > + * Version to be reported to the VPU
> > > + * VPU assumes 0 (aka 1) which does not require the released
> > > callback, nor
> > > + * expect the client to handle VC_MEM_REQUESTS.
> > > + * Version 2 requires the released callback, and must support
> > > VC_MEM_REQUESTS.
> > > + */
> > > +#define VC_SM_PROTOCOL_VERSION 2
> > > +
> > > +enum vc_sm_msg_type {
> > > + /* Message types supported for HOST->VC direction */
> > > +
> > > + /* Allocate shared memory block */
> > > + VC_SM_MSG_TYPE_ALLOC,
> > > + /* Lock allocated shared memory block */
> > > + VC_SM_MSG_TYPE_LOCK,
> > > + /* Unlock allocated shared memory block */
> > > + VC_SM_MSG_TYPE_UNLOCK,
> > > + /* Unlock allocated shared memory block, do not answer command
> > > */
> > > + VC_SM_MSG_TYPE_UNLOCK_NOANS,
> > > + /* Free shared memory block */
> > > + VC_SM_MSG_TYPE_FREE,
> > > + /* Resize a shared memory block */
> > > + VC_SM_MSG_TYPE_RESIZE,
> > > + /* Walk the allocated shared memory block(s) */
> > > + VC_SM_MSG_TYPE_WALK_ALLOC,
> > > +
> > > + /* A previously applied action will need to be reverted */
> > > + VC_SM_MSG_TYPE_ACTION_CLEAN,
> > > +
> > > + /*
> > > + * Import a physical address and wrap into a MEM_HANDLE_T.
> > > + * Release with VC_SM_MSG_TYPE_FREE.
> > > + */
> > > + VC_SM_MSG_TYPE_IMPORT,
> > > + /*
> > > + *Tells VC the protocol version supported by this client.
> > > + * 2 supports the async/cmd messages from the VPU for final
> > > release
> > > + * of memory, and for VC allocations.
> > > + */
> > > + VC_SM_MSG_TYPE_CLIENT_VERSION,
> > > + /* Response to VC request for memory */
> > > + VC_SM_MSG_TYPE_VC_MEM_REQUEST_REPLY,
> > > +
> > > + /*
> > > + * Asynchronous/cmd messages supported for VC->HOST direction.
> > > + * Signalled by setting the top bit in vc_sm_result_t trans_id.
> > > + */
> > > +
> > > + /*
> > > + * VC has finished with an imported memory allocation.
> > > + * Release any Linux reference counts on the underlying block.
> > > + */
> > > + VC_SM_MSG_TYPE_RELEASED,
> > > + /* VC request for memory */
> > > + VC_SM_MSG_TYPE_VC_MEM_REQUEST,
> > > +
> > > + VC_SM_MSG_TYPE_MAX
> > > +};
> > > +
> > > +/* Type of memory to be allocated */
> > > +enum vc_sm_alloc_type_t {
> > > + VC_SM_ALLOC_CACHED,
> > > + VC_SM_ALLOC_NON_CACHED,
> > > +};
> > > +
> > > +/* Message header for all messages in HOST->VC direction */
> > > +struct vc_sm_msg_hdr_t {
> > > + u32 type;
> > > + u32 trans_id;
> > > + u8 body[0];
> > > +
> > > +};
> > > +
> > > +/* Request to allocate memory (HOST->VC) */
> > > +struct vc_sm_alloc_t {
> > > + /* type of memory to allocate */
> > > + enum vc_sm_alloc_type_t type;
> > > + /* byte amount of data to allocate per unit */
> > > + u32 base_unit;
> > > + /* number of unit to allocate */
> > > + u32 num_unit;
> > > + /* alignment to be applied on allocation */
> > > + u32 alignment;
> > > + /* identity of who allocated this block */
> > > + u32 allocator;
> > > + /* resource name (for easier tracking on vc side) */
> > > + char name[VC_SM_RESOURCE_NAME];
> > > +
> > > +};
> > > +
> > > +/* Result of a requested memory allocation (VC->HOST) */
> > > +struct vc_sm_alloc_result_t {
> > > + /* Transaction identifier */
> > > + u32 trans_id;
> > > +
> > > + /* Resource handle */
> > > + u32 res_handle;
> > > + /* Pointer to resource buffer */
> > > + u32 res_mem;
> > > + /* Resource base size (bytes) */
> > > + u32 res_base_size;
> > > + /* Resource number */
> > > + u32 res_num;
> > > +
> > > +};
> > > +
> > > +/* Request to free a previously allocated memory (HOST->VC) */
> > > +struct vc_sm_free_t {
> > > + /* Resource handle (returned from alloc) */
> > > + u32 res_handle;
> > > + /* Resource buffer (returned from alloc) */
> > > + u32 res_mem;
> > > +
> > > +};
> > > +
> > > +/* Request to lock a previously allocated memory (HOST->VC) */
> > > +struct vc_sm_lock_unlock_t {
> > > + /* Resource handle (returned from alloc) */
> > > + u32 res_handle;
> > > + /* Resource buffer (returned from alloc) */
> > > + u32 res_mem;
> > > +
> > > +};
> > > +
> > > +/* Request to resize a previously allocated memory (HOST->VC) */
> > > +struct vc_sm_resize_t {
> > > + /* Resource handle (returned from alloc) */
> > > + u32 res_handle;
> > > + /* Resource buffer (returned from alloc) */
> > > + u32 res_mem;
> > > + /* Resource *new* size requested (bytes) */
> > > + u32 res_new_size;
> > > +
> > > +};
> > > +
> > > +/* Result of a requested memory lock (VC->HOST) */
> > > +struct vc_sm_lock_result_t {
> > > + /* Transaction identifier */
> > > + u32 trans_id;
> > > +
> > > + /* Resource handle */
> > > + u32 res_handle;
> > > + /* Pointer to resource buffer */
> > > + u32 res_mem;
> > > + /*
> > > + * Pointer to former resource buffer if the memory
> > > + * was reallocated
> > > + */
> > > + u32 res_old_mem;
> > > +
> > > +};
> > > +
> > > +/* Generic result for a request (VC->HOST) */
> > > +struct vc_sm_result_t {
> > > + /* Transaction identifier */
> > > + u32 trans_id;
> > > +
> > > + s32 success;
> > > +
> > > +};
> > > +
> > > +/* Request to revert a previously applied action (HOST->VC) */
> > > +struct vc_sm_action_clean_t {
> > > + /* Action of interest */
> > > + enum vc_sm_msg_type res_action;
> > > + /* Transaction identifier for the action of interest */
> > > + u32 action_trans_id;
> > > +
> > > +};
> > > +
> > > +/* Request to remove all data associated with a given allocator
> > > (HOST->VC) */
> > > +struct vc_sm_free_all_t {
> > > + /* Allocator identifier */
> > > + u32 allocator;
> > > +};
> > > +
> > > +/* Request to import memory (HOST->VC) */
> > > +struct vc_sm_import {
> > > + /* type of memory to allocate */
> > > + enum vc_sm_alloc_type_t type;
> > > + /* pointer to the VC (ie physical) address of the allocated
> > > memory */
> > > + u32 addr;
> > > + /* size of buffer */
> > > + u32 size;
> > > + /* opaque handle returned in RELEASED messages */
> > > + u32 kernel_id;
> > > + /* Allocator identifier */
> > > + u32 allocator;
> > > + /* resource name (for easier tracking on vc side) */
> > > + char name[VC_SM_RESOURCE_NAME];
> > > +};
> > > +
> > > +/* Result of a requested memory import (VC->HOST) */
> > > +struct vc_sm_import_result {
> > > + /* Transaction identifier */
> > > + u32 trans_id;
> > > +
> > > + /* Resource handle */
> > > + u32 res_handle;
> > > +};
> > > +
> > > +/* Notification that VC has finished with an allocation (VC->HOST)
> > > */
> > > +struct vc_sm_released {
> > > + /* cmd type / trans_id */
> > > + u32 cmd;
> > > +
> > > + /* pointer to the VC (ie physical) address of the allocated
> > > memory */
> > > + u32 addr;
> > > + /* size of buffer */
> > > + u32 size;
> > > + /* opaque handle returned in RELEASED messages */
> > > + u32 kernel_id;
> > > + u32 vc_handle;
> > > +};
> > > +
> > > +/*
> > > + * Client informing VC as to the protocol version it supports.
> > > + * >=2 requires the released callback, and supports VC asking for
> > > memory.
> > > + * Failure means that the firmware doesn't support this call, and
> > > therefore the
> > > + * client should either fail, or NOT rely on getting the released
> > > callback.
> > > + */
> > > +struct vc_sm_version {
> > > + u32 version;
> > > +};
> > > +
> > > +/* Request FROM VideoCore for some memory */
> > > +struct vc_sm_vc_mem_request {
> > > + /* cmd type */
> > > + u32 cmd;
> > > +
> > > + /* trans_id (from VPU) */
> > > + u32 trans_id;
> > > + /* size of buffer */
> > > + u32 size;
> > > + /* alignment of buffer */
> > > + u32 align;
> > > + /* resource name (for easier tracking) */
> > > + char name[VC_SM_RESOURCE_NAME];
> > > + /* VPU handle for the resource */
> > > + u32 vc_handle;
> > > +};
> > > +
> > > +/* Response from the kernel to provide the VPU with some memory */
> > > +struct vc_sm_vc_mem_request_result {
> > > + /* Transaction identifier for the VPU */
> > > + u32 trans_id;
> > > + /* pointer to the physical address of the allocated memory */
> > > + u32 addr;
> > > + /* opaque handle returned in RELEASED messages */
> > > + u32 kernel_id;
> > > +};
> > > +
> > > +/* Union of ALL messages */
> > > +union vc_sm_msg_union_t {
> > > + struct vc_sm_alloc_t alloc;
> > > + struct vc_sm_alloc_result_t alloc_result;
> > > + struct vc_sm_free_t free;
> > > + struct vc_sm_lock_unlock_t lock_unlock;
> > > + struct vc_sm_action_clean_t action_clean;
> > > + struct vc_sm_resize_t resize;
> > > + struct vc_sm_lock_result_t lock_result;
> > > + struct vc_sm_result_t result;
> > > + struct vc_sm_free_all_t free_all;
> > > + struct vc_sm_import import;
> > > + struct vc_sm_import_result import_result;
> > > + struct vc_sm_version version;
> > > + struct vc_sm_released released;
> > > + struct vc_sm_vc_mem_request vc_request;
> > > + struct vc_sm_vc_mem_request_result vc_request_result;
> > > +};
> > > +
> > > +#endif /* __VC_SM_DEFS_H__INCLUDED__ */
> > > diff --git a/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> > > b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> > > new file mode 100644
> > > index 000000000000..988fdd967922
> > > --- /dev/null
> > > +++ b/drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> > > @@ -0,0 +1,28 @@
> > > +/* SPDX-License-Identifier: GPL-2.0 */
> > > +
> > > +/*
> > > + * VideoCore Shared Memory CMA allocator
> > > + *
> > > + * Copyright: 2018, Raspberry Pi (Trading) Ltd
> > > + *
> > > + * Based on vc_sm_defs.h from the vmcs_sm driver Copyright Broadcom
> > > Corporation.
> > > + *
> > > + */
> > > +
> > > +#ifndef __VC_SM_KNL_H__INCLUDED__
> > > +#define __VC_SM_KNL_H__INCLUDED__
> > > +
> > > +#if !defined(__KERNEL__)
> > > +#error "This interface is for kernel use only..."
> > > +#endif
> > > +
> > > +/* Free a previously allocated or imported shared memory handle and
> > > block. */
> > > +int vc_sm_cma_free(void *handle);
> > > +
> > > +/* Get an internal resource handle mapped from the external one. */
> > > +int vc_sm_cma_int_handle(void *handle);
> > > +
> > > +/* Import a block of memory into the GPU space. */
> > > +int vc_sm_cma_import_dmabuf(struct dma_buf *dmabuf, void **handle);
> > > +
> > > +#endif /* __VC_SM_KNL_H__INCLUDED__ */
> >
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-08-24 16:39 ` Jacopo Mondi
@ 2020-08-25 17:52 ` Dave Stevenson
2020-08-27 10:38 ` Jacopo Mondi
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-08-25 17:52 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Nicolas Saenz Julienne, Laurent Pinchart,
Linux Media Mailing List, Dave Stevenson, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
Hi Jacopo
On Mon, 24 Aug 2020 at 17:36, Jacopo Mondi <jacopo@jmondi.org> wrote:
>
> Hi Dave, Nicolas, Laurent,
>
> On Wed, May 06, 2020 at 08:24:38PM +0100, Dave Stevenson wrote:
> > Hi Nicolas
> >
> > On Wed, 6 May 2020 at 19:04, Nicolas Saenz Julienne
> > <nsaenzjulienne@suse.de> wrote:
> > >
> > > Hi Laurent, Dave,
> > >
> > > On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > > > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > > >
> > > > Add Broadcom VideoCore Shared Memory support.
> > > >
> > > > This new driver allows contiguous memory blocks to be imported
> > > > into the VideoCore VPU memory map, and manages the lifetime of
> > > > those objects, only releasing the source dmabuf once the VPU has
> > > > confirmed it has finished with it.
> > > >
> > >
> > > I'm still digesting all this, but a question came up, who is using the
> > > ioctls?
> >
> > We have a userspace library that uses it [1].
> > It is used by things like MMAL to share buffers between the VPU and
> > ARM, rather than having to get VCHI to copy all the data between
> > mirrored buffers.
> >
> > I think what has happened here is that Laurent has picked up the
> > version of the driver from the top of our downstream kernel tree.
> > For libcamera and the ISP driver, we need a significantly smaller
> > feature set, basically import of dmabufs only, no allocations or cache
> > management. For the ISP driver it's mainly dmabuf import from
> > videobuf2 for the image buffers, but there's also a need to pass in
> > lens shading tables which are relatively large. With a small amount of
> > rework in libcamera, we can make it so that we use dma-buf heaps to do
> > the allocation, and pass in a dmabuf fd to the ISP driver to then map
> > onto the VPU. That removes all the ioctls handling from this driver.
> >
> > Downstream we do have other use cases that want to be able to do other
> > functions on shared memory, but that too should be reworkable into
> > using dma-buf heaps for allocations, and vcsm only handles importing
> > dmabufs via an ioctl. All that can be hidden away in the vcsm library,
> > so applications don't care.
> > We've also got some legacy code kicking around, as there was
> > originally a version of the driver that mapped the VPU's memory blocks
> > to the ARM. That's why the vcsm library has two code paths through
> > almost every function - one for each driver.
> >
> > Laurent: What's your view? Halt the review this particular patch for
> > now and rework, or try and get this all integrated?
> > Mainline obviously already has dma-buf heaps merged, whilst I have a
> > PR cherry-picking it back into our downstream 5.4. The main reason it
> > hasn't been merged is that I haven't had a test case to prove it
> > works. The rework should be relatively simple, but will need small
> > updates to both libcamera and ISP driver.
>
> As months have passed, libcamera moved to allocate lens shading tables
> using dma-buf heaps and the only user I can name of the vc-sm-cma
> driver is the actual ISP, that needs to import the dmabuf pointing to
> the lens shading maps with vc_sm_cma_import_dmabuf().
You've also got vc04_services/vchiq-mmal/mmal-vchiq.c importing
dmabufs, either from vb2_contig or imported from elsewhere when using
VB2_MEMORY_DMABUF.
> Upstreaming the whole vc-sm-cma driver as it is for this single kAPI
> seems a bit a no-go. Dave, what would you prefer here ? Should I
> provide a minimal vc-sm-cam driver that only performs buffer importing
> to support the ISP driver ? Is the buffer importing into VPU there to
> stay or is its usage transitional and can be kept out of the next
> submission of this series ?
Both imports are here to stay as the VPU needs to be able to use those
blocks of memory.
This first iteration picked up a fair number of extraneous lumps (eg
the caching calls).
I got a reminder last week that I promised a reworked version of
vc-sm-cma to you and I hadn't done it - sorry, juggling too many
things. I'll get on it now, so nudge me if I haven't pushed it to you
by the end of the week for your review.
We can trim it down significantly now that we have dma-heaps in and
working. There's a niggle that the current dma-heaps are always cached
on the ARM, but that just means that the user has to be careful to use
DMA_BUF_IOCTL_SYNC correctly (which they should be doing anyway).
Whilst waiting for that, the Unicam driver, and the prep work in
mmal-vchiq could all be pushed first, and ideally as two independent
patchsets as there are no inter-dependencies between them.
Dave
> Thanks
> j
>
> >
> > Dave
> >
> > [1] https://github.com/raspberrypi/userland/tree/master/host_applications/linux/libs/sm
> >
> > > Regards,
> > > Nicolas
> > >
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-08-25 17:52 ` Dave Stevenson
@ 2020-08-27 10:38 ` Jacopo Mondi
2020-08-27 12:51 ` Dave Stevenson
0 siblings, 1 reply; 111+ messages in thread
From: Jacopo Mondi @ 2020-08-27 10:38 UTC (permalink / raw)
To: Dave Stevenson
Cc: Nicolas Saenz Julienne, Laurent Pinchart,
Linux Media Mailing List, Dave Stevenson, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
Hi Dave,
On Tue, Aug 25, 2020 at 06:52:18PM +0100, Dave Stevenson wrote:
> Hi Jacopo
>
> On Mon, 24 Aug 2020 at 17:36, Jacopo Mondi <jacopo@jmondi.org> wrote:
> >
> > Hi Dave, Nicolas, Laurent,
> >
> > On Wed, May 06, 2020 at 08:24:38PM +0100, Dave Stevenson wrote:
> > > Hi Nicolas
> > >
> > > On Wed, 6 May 2020 at 19:04, Nicolas Saenz Julienne
> > > <nsaenzjulienne@suse.de> wrote:
> > > >
> > > > Hi Laurent, Dave,
> > > >
> > > > On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > > > > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > > > >
> > > > > Add Broadcom VideoCore Shared Memory support.
> > > > >
> > > > > This new driver allows contiguous memory blocks to be imported
> > > > > into the VideoCore VPU memory map, and manages the lifetime of
> > > > > those objects, only releasing the source dmabuf once the VPU has
> > > > > confirmed it has finished with it.
> > > > >
> > > >
> > > > I'm still digesting all this, but a question came up, who is using the
> > > > ioctls?
> > >
> > > We have a userspace library that uses it [1].
> > > It is used by things like MMAL to share buffers between the VPU and
> > > ARM, rather than having to get VCHI to copy all the data between
> > > mirrored buffers.
> > >
> > > I think what has happened here is that Laurent has picked up the
> > > version of the driver from the top of our downstream kernel tree.
> > > For libcamera and the ISP driver, we need a significantly smaller
> > > feature set, basically import of dmabufs only, no allocations or cache
> > > management. For the ISP driver it's mainly dmabuf import from
> > > videobuf2 for the image buffers, but there's also a need to pass in
> > > lens shading tables which are relatively large. With a small amount of
> > > rework in libcamera, we can make it so that we use dma-buf heaps to do
> > > the allocation, and pass in a dmabuf fd to the ISP driver to then map
> > > onto the VPU. That removes all the ioctls handling from this driver.
> > >
> > > Downstream we do have other use cases that want to be able to do other
> > > functions on shared memory, but that too should be reworkable into
> > > using dma-buf heaps for allocations, and vcsm only handles importing
> > > dmabufs via an ioctl. All that can be hidden away in the vcsm library,
> > > so applications don't care.
> > > We've also got some legacy code kicking around, as there was
> > > originally a version of the driver that mapped the VPU's memory blocks
> > > to the ARM. That's why the vcsm library has two code paths through
> > > almost every function - one for each driver.
> > >
> > > Laurent: What's your view? Halt the review this particular patch for
> > > now and rework, or try and get this all integrated?
> > > Mainline obviously already has dma-buf heaps merged, whilst I have a
> > > PR cherry-picking it back into our downstream 5.4. The main reason it
> > > hasn't been merged is that I haven't had a test case to prove it
> > > works. The rework should be relatively simple, but will need small
> > > updates to both libcamera and ISP driver.
> >
> > As months have passed, libcamera moved to allocate lens shading tables
> > using dma-buf heaps and the only user I can name of the vc-sm-cma
> > driver is the actual ISP, that needs to import the dmabuf pointing to
> > the lens shading maps with vc_sm_cma_import_dmabuf().
>
> You've also got vc04_services/vchiq-mmal/mmal-vchiq.c importing
> dmabufs, either from vb2_contig or imported from elsewhere when using
> VB2_MEMORY_DMABUF.
Of course. Re-looking at it, the lens-shading tables are allocated on
dmabuf heaps and the exported dmabuf fd passed with a custom control to the
ISP, which uses it to set a mmal port parameter. I got lost in the code
base at mmal-vchiq.c:port_parameter_set(), which receives a
struct bcm2835_isp_lens_shading which contains the dmabuf fd. I assume
it then maps it into the VPU memory to access the shading tables.
But of course buffer queueing to the ISP requires dmabuf importing in
the VPU, and that happens by 'submitting' a buffer to mmal-vchiq
vchiq_mmal_submit_buffer() which does that by calling
vc_sm_cma_import_dmabuf().
I hope I have a more clear idea of the two paths now.
>
> > Upstreaming the whole vc-sm-cma driver as it is for this single kAPI
> > seems a bit a no-go. Dave, what would you prefer here ? Should I
> > provide a minimal vc-sm-cam driver that only performs buffer importing
> > to support the ISP driver ? Is the buffer importing into VPU there to
> > stay or is its usage transitional and can be kept out of the next
> > submission of this series ?
>
> Both imports are here to stay as the VPU needs to be able to use those
> blocks of memory.
>
Of course. I was wondering if a fairly big component like vc-sma-cma
isn't too much for just importing, and reading further it seems like
this is a shared concern.
> This first iteration picked up a fair number of extraneous lumps (eg
> the caching calls).
> I got a reminder last week that I promised a reworked version of
> vc-sm-cma to you and I hadn't done it - sorry, juggling too many
> things. I'll get on it now, so nudge me if I haven't pushed it to you
> by the end of the week for your review.
Great, so I'll wait for news from your side
>
> We can trim it down significantly now that we have dma-heaps in and
> working. There's a niggle that the current dma-heaps are always cached
> on the ARM, but that just means that the user has to be careful to use
> DMA_BUF_IOCTL_SYNC correctly (which they should be doing anyway).
I am running what was meant to be a v2 of this series and I get a
complaint:
vc_sm_cma_import_dmabuf_internal: Expecting an uncached alias for dma_addr
When I read this paragraph from your email yesterday I immediately
thought "this should be the LSC table".
Debugging it further I found out it's actually a vb2 buffer. I have
reduced the list of patches in v2 compared to this long one, and I'm
probably left out something relevant :/
>
> Whilst waiting for that, the Unicam driver, and the prep work in
> mmal-vchiq could all be pushed first, and ideally as two independent
> patchsets as there are no inter-dependencies between them.
I could start sending out the unicam driver, yes.
Currently I'm a bit stuck not being able to receive frames from the
unicam driver. I see the buffers being returned by the ISR routine,
but I never get a buffer available notification in libcamera.
There's been a few changes to the downstream unicam driver (ie
requesting the VPU clock frequency) and I see the RPi mainline support
has moved forward quite a bit since v5.8. Are you aware of any trivial
change I might be missing that could cause this ?
Thanks
j
>
> Dave
>
> > Thanks
> > j
> >
> > >
> > > Dave
> > >
> > > [1] https://github.com/raspberrypi/userland/tree/master/host_applications/linux/libs/sm
> > >
> > > > Regards,
> > > > Nicolas
> > > >
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-08-27 10:38 ` Jacopo Mondi
@ 2020-08-27 12:51 ` Dave Stevenson
2020-08-27 16:46 ` Jacopo Mondi
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-08-27 12:51 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Nicolas Saenz Julienne, Laurent Pinchart,
Linux Media Mailing List, Dave Stevenson, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
On Thu, 27 Aug 2020 at 11:34, Jacopo Mondi <jacopo@jmondi.org> wrote:
>
> Hi Dave,
>
> On Tue, Aug 25, 2020 at 06:52:18PM +0100, Dave Stevenson wrote:
> > Hi Jacopo
> >
> > On Mon, 24 Aug 2020 at 17:36, Jacopo Mondi <jacopo@jmondi.org> wrote:
> > >
> > > Hi Dave, Nicolas, Laurent,
> > >
> > > On Wed, May 06, 2020 at 08:24:38PM +0100, Dave Stevenson wrote:
> > > > Hi Nicolas
> > > >
> > > > On Wed, 6 May 2020 at 19:04, Nicolas Saenz Julienne
> > > > <nsaenzjulienne@suse.de> wrote:
> > > > >
> > > > > Hi Laurent, Dave,
> > > > >
> > > > > On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > > > > > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > > > > >
> > > > > > Add Broadcom VideoCore Shared Memory support.
> > > > > >
> > > > > > This new driver allows contiguous memory blocks to be imported
> > > > > > into the VideoCore VPU memory map, and manages the lifetime of
> > > > > > those objects, only releasing the source dmabuf once the VPU has
> > > > > > confirmed it has finished with it.
> > > > > >
> > > > >
> > > > > I'm still digesting all this, but a question came up, who is using the
> > > > > ioctls?
> > > >
> > > > We have a userspace library that uses it [1].
> > > > It is used by things like MMAL to share buffers between the VPU and
> > > > ARM, rather than having to get VCHI to copy all the data between
> > > > mirrored buffers.
> > > >
> > > > I think what has happened here is that Laurent has picked up the
> > > > version of the driver from the top of our downstream kernel tree.
> > > > For libcamera and the ISP driver, we need a significantly smaller
> > > > feature set, basically import of dmabufs only, no allocations or cache
> > > > management. For the ISP driver it's mainly dmabuf import from
> > > > videobuf2 for the image buffers, but there's also a need to pass in
> > > > lens shading tables which are relatively large. With a small amount of
> > > > rework in libcamera, we can make it so that we use dma-buf heaps to do
> > > > the allocation, and pass in a dmabuf fd to the ISP driver to then map
> > > > onto the VPU. That removes all the ioctls handling from this driver.
> > > >
> > > > Downstream we do have other use cases that want to be able to do other
> > > > functions on shared memory, but that too should be reworkable into
> > > > using dma-buf heaps for allocations, and vcsm only handles importing
> > > > dmabufs via an ioctl. All that can be hidden away in the vcsm library,
> > > > so applications don't care.
> > > > We've also got some legacy code kicking around, as there was
> > > > originally a version of the driver that mapped the VPU's memory blocks
> > > > to the ARM. That's why the vcsm library has two code paths through
> > > > almost every function - one for each driver.
> > > >
> > > > Laurent: What's your view? Halt the review this particular patch for
> > > > now and rework, or try and get this all integrated?
> > > > Mainline obviously already has dma-buf heaps merged, whilst I have a
> > > > PR cherry-picking it back into our downstream 5.4. The main reason it
> > > > hasn't been merged is that I haven't had a test case to prove it
> > > > works. The rework should be relatively simple, but will need small
> > > > updates to both libcamera and ISP driver.
> > >
> > > As months have passed, libcamera moved to allocate lens shading tables
> > > using dma-buf heaps and the only user I can name of the vc-sm-cma
> > > driver is the actual ISP, that needs to import the dmabuf pointing to
> > > the lens shading maps with vc_sm_cma_import_dmabuf().
> >
> > You've also got vc04_services/vchiq-mmal/mmal-vchiq.c importing
> > dmabufs, either from vb2_contig or imported from elsewhere when using
> > VB2_MEMORY_DMABUF.
>
> Of course. Re-looking at it, the lens-shading tables are allocated on
> dmabuf heaps and the exported dmabuf fd passed with a custom control to the
> ISP, which uses it to set a mmal port parameter. I got lost in the code
> base at mmal-vchiq.c:port_parameter_set(), which receives a
> struct bcm2835_isp_lens_shading which contains the dmabuf fd. I assume
> it then maps it into the VPU memory to access the shading tables.
>
> But of course buffer queueing to the ISP requires dmabuf importing in
> the VPU, and that happens by 'submitting' a buffer to mmal-vchiq
> vchiq_mmal_submit_buffer() which does that by calling
> vc_sm_cma_import_dmabuf().
>
> I hope I have a more clear idea of the two paths now.
>
> >
> > > Upstreaming the whole vc-sm-cma driver as it is for this single kAPI
> > > seems a bit a no-go. Dave, what would you prefer here ? Should I
> > > provide a minimal vc-sm-cam driver that only performs buffer importing
> > > to support the ISP driver ? Is the buffer importing into VPU there to
> > > stay or is its usage transitional and can be kept out of the next
> > > submission of this series ?
> >
> > Both imports are here to stay as the VPU needs to be able to use those
> > blocks of memory.
> >
>
> Of course. I was wondering if a fairly big component like vc-sma-cma
> isn't too much for just importing, and reading further it seems like
> this is a shared concern.
>
> > This first iteration picked up a fair number of extraneous lumps (eg
> > the caching calls).
> > I got a reminder last week that I promised a reworked version of
> > vc-sm-cma to you and I hadn't done it - sorry, juggling too many
> > things. I'll get on it now, so nudge me if I haven't pushed it to you
> > by the end of the week for your review.
>
> Great, so I'll wait for news from your side
>
> >
> > We can trim it down significantly now that we have dma-heaps in and
> > working. There's a niggle that the current dma-heaps are always cached
> > on the ARM, but that just means that the user has to be careful to use
> > DMA_BUF_IOCTL_SYNC correctly (which they should be doing anyway).
>
> I am running what was meant to be a v2 of this series and I get a
> complaint:
> vc_sm_cma_import_dmabuf_internal: Expecting an uncached alias for dma_addr
>
> When I read this paragraph from your email yesterday I immediately
> thought "this should be the LSC table".
>
> Debugging it further I found out it's actually a vb2 buffer. I have
> reduced the list of patches in v2 compared to this long one, and I'm
> probably left out something relevant :/
vb2 allocated by the ISP (VB2_MEMORY_MMAP), or from Unicam that is
then imported into the ISP (VB2_MEMORY_DMABUF)?
The former is the dma-ranges being incorrectly set, and the reason for
the patch where VCHIQ children inherit the parent's dma config. It's
possible something else has changed in the setup since.
The latter is likely to be more involved and depend on what actually
allocated it (which may mean I'm using the wrong API calls).
I'm getting the dma address via sg_dma_address having attached and
mapped the dmabuf. I'm expecting that dma address to therefore follow
the dma-ranges of my device (not the allocator), but I'm not 100%
certain that is what does happen.
A few more details of what exactly the use case that triggers it would
be useful.
> >
> > Whilst waiting for that, the Unicam driver, and the prep work in
> > mmal-vchiq could all be pushed first, and ideally as two independent
> > patchsets as there are no inter-dependencies between them.
>
> I could start sending out the unicam driver, yes.
>
> Currently I'm a bit stuck not being able to receive frames from the
> unicam driver. I see the buffers being returned by the ISR routine,
> but I never get a buffer available notification in libcamera.
>
> There's been a few changes to the downstream unicam driver (ie
> requesting the VPU clock frequency) and I see the RPi mainline support
> has moved forward quite a bit since v5.8. Are you aware of any trivial
> change I might be missing that could cause this ?
Things never stand still!
Requesting the VPU clock is to avoid a FIFO overflow.
We've gained pixel format support (with defines and docs) for Y12P and Y14P.
And hopefully I fixed up all the review comments from v1.
It's hard to guess why libcamera isn't happy if we're getting ISRs. We
do need to get the appropriate flags in the ISR for frame end (FEI or
PI0).
Is this Pi3 or Pi4 that you're testing on? Pi3 support should be sound
on mainline. AFAIK Pi4 is still a work in progress.
Can you stream just using v4l2-ctl --stream-mmap=3 --stream-count=1000
--stream-to=/dev/null ? That should work with /dev/video0. Getting the
embedded data is more involved over the ordering of opening and
starting streaming on the nodes.
(We do have an open issue regarding getting the correct DMA address
for Unicam on PI0 & 1. Those share the L2 cache for many things so
dma-ranges is set differently, and not in a way that currently works
with the Unicam driver. That one may need a followup fix.)
Dave
> Thanks
> j
>
> >
> > Dave
> >
> > > Thanks
> > > j
> > >
> > > >
> > > > Dave
> > > >
> > > > [1] https://github.com/raspberrypi/userland/tree/master/host_applications/linux/libs/sm
> > > >
> > > > > Regards,
> > > > > Nicolas
> > > > >
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-08-27 12:51 ` Dave Stevenson
@ 2020-08-27 16:46 ` Jacopo Mondi
2020-08-27 17:19 ` Dave Stevenson
0 siblings, 1 reply; 111+ messages in thread
From: Jacopo Mondi @ 2020-08-27 16:46 UTC (permalink / raw)
To: Dave Stevenson
Cc: Nicolas Saenz Julienne, Laurent Pinchart,
Linux Media Mailing List, Dave Stevenson, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
Hi Dave,
On Thu, Aug 27, 2020 at 01:51:07PM +0100, Dave Stevenson wrote:
> On Thu, 27 Aug 2020 at 11:34, Jacopo Mondi <jacopo@jmondi.org> wrote:
[snip]
> > >
> > > We can trim it down significantly now that we have dma-heaps in and
> > > working. There's a niggle that the current dma-heaps are always cached
> > > on the ARM, but that just means that the user has to be careful to use
> > > DMA_BUF_IOCTL_SYNC correctly (which they should be doing anyway).
> >
> > I am running what was meant to be a v2 of this series and I get a
> > complaint:
> > vc_sm_cma_import_dmabuf_internal: Expecting an uncached alias for dma_addr
> >
> > When I read this paragraph from your email yesterday I immediately
> > thought "this should be the LSC table".
> >
> > Debugging it further I found out it's actually a vb2 buffer. I have
> > reduced the list of patches in v2 compared to this long one, and I'm
> > probably left out something relevant :/
>
> vb2 allocated by the ISP (VB2_MEMORY_MMAP), or from Unicam that is
> then imported into the ISP (VB2_MEMORY_DMABUF)?
>
> The former is the dma-ranges being incorrectly set, and the reason for
> the patch where VCHIQ children inherit the parent's dma config. It's
> possible something else has changed in the setup since.
Re-introducing those two patches which I left out yesterday waiting
for more comments made the warning go away, so I presume that was
memory allocated on the ISP, exported as dmabuf by libcamera, and then
re-imported in the video device (that's what our FrameBufferAllocator
does, and I was testing with 'cam' which uses that class to reserve
memory per-stream).
I have a discussion open with Phil and Nicolas to see how those two
patches might be made upstream consumable, but I'll remember to keep
them in when testing.
>
> The latter is likely to be more involved and depend on what actually
> allocated it (which may mean I'm using the wrong API calls).
> I'm getting the dma address via sg_dma_address having attached and
> mapped the dmabuf. I'm expecting that dma address to therefore follow
> the dma-ranges of my device (not the allocator), but I'm not 100%
> certain that is what does happen.
>
> A few more details of what exactly the use case that triggers it would
> be useful.
>
> > >
> > > Whilst waiting for that, the Unicam driver, and the prep work in
> > > mmal-vchiq could all be pushed first, and ideally as two independent
> > > patchsets as there are no inter-dependencies between them.
> >
> > I could start sending out the unicam driver, yes.
> >
> > Currently I'm a bit stuck not being able to receive frames from the
> > unicam driver. I see the buffers being returned by the ISR routine,
> > but I never get a buffer available notification in libcamera.
> >
> > There's been a few changes to the downstream unicam driver (ie
> > requesting the VPU clock frequency) and I see the RPi mainline support
> > has moved forward quite a bit since v5.8. Are you aware of any trivial
> > change I might be missing that could cause this ?
>
> Things never stand still!
Isn't it great ? :D
> Requesting the VPU clock is to avoid a FIFO overflow.
> We've gained pixel format support (with defines and docs) for Y12P and Y14P.
> And hopefully I fixed up all the review comments from v1.
Do you think it's worth upstreaming those parts in v2 or should it be
done on top ? I should check if the VPU clock is exposed or not in
mainline first...
>
> It's hard to guess why libcamera isn't happy if we're getting ISRs. We
> do need to get the appropriate flags in the ISR for frame end (FEI or
> PI0).
> Is this Pi3 or Pi4 that you're testing on? Pi3 support should be sound
> on mainline. AFAIK Pi4 is still a work in progress.
> Can you stream just using v4l2-ctl --stream-mmap=3 --stream-count=1000
> --stream-to=/dev/null ? That should work with /dev/video0. Getting the
> embedded data is more involved over the ordering of opening and
> starting streaming on the nodes.
It's a Pi4.
After a pleasant day of debugging I got a nice trace that shows me
that it's actually the buffer importing part that hangs on a completion
in vc_sm_cma_vchi_import.
In mainline the whole vchi interface got dropped and I got to port the
vc-sma-cma driver to use the vchiq interface directly. I might have
screwed something up and I'm now looking suspiciously at this thread
function: "vc_sm_cma_vchi_videocore_io()" (also because I had to move it
to use msg_hold+msg_release, as peek+remove was killed by:
b5f1547b6e3bd ("staging: vchi: Get rid of vchi_msg_peek()")
I'll look into that again.
In the meantime is there any chance this rings any bell to you?
https://paste.debian.net/1161469/
Thanks
j
>
> (We do have an open issue regarding getting the correct DMA address
> for Unicam on PI0 & 1. Those share the L2 cache for many things so
> dma-ranges is set differently, and not in a way that currently works
> with the Unicam driver. That one may need a followup fix.)
>
> Dave
>
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-08-27 16:46 ` Jacopo Mondi
@ 2020-08-27 17:19 ` Dave Stevenson
0 siblings, 0 replies; 111+ messages in thread
From: Dave Stevenson @ 2020-08-27 17:19 UTC (permalink / raw)
To: Jacopo Mondi
Cc: Nicolas Saenz Julienne, Laurent Pinchart,
Linux Media Mailing List, Dave Stevenson, Kieran Bingham,
Niklas Söderlund, Naushir Patuck
Hi Jacopo
On Thu, 27 Aug 2020 at 17:42, Jacopo Mondi <jacopo@jmondi.org> wrote:
>
> Hi Dave,
>
> On Thu, Aug 27, 2020 at 01:51:07PM +0100, Dave Stevenson wrote:
> > On Thu, 27 Aug 2020 at 11:34, Jacopo Mondi <jacopo@jmondi.org> wrote:
>
> [snip]
>
> > > >
> > > > We can trim it down significantly now that we have dma-heaps in and
> > > > working. There's a niggle that the current dma-heaps are always cached
> > > > on the ARM, but that just means that the user has to be careful to use
> > > > DMA_BUF_IOCTL_SYNC correctly (which they should be doing anyway).
> > >
> > > I am running what was meant to be a v2 of this series and I get a
> > > complaint:
> > > vc_sm_cma_import_dmabuf_internal: Expecting an uncached alias for dma_addr
> > >
> > > When I read this paragraph from your email yesterday I immediately
> > > thought "this should be the LSC table".
> > >
> > > Debugging it further I found out it's actually a vb2 buffer. I have
> > > reduced the list of patches in v2 compared to this long one, and I'm
> > > probably left out something relevant :/
> >
> > vb2 allocated by the ISP (VB2_MEMORY_MMAP), or from Unicam that is
> > then imported into the ISP (VB2_MEMORY_DMABUF)?
> >
> > The former is the dma-ranges being incorrectly set, and the reason for
> > the patch where VCHIQ children inherit the parent's dma config. It's
> > possible something else has changed in the setup since.
>
> Re-introducing those two patches which I left out yesterday waiting
> for more comments made the warning go away, so I presume that was
> memory allocated on the ISP, exported as dmabuf by libcamera, and then
> re-imported in the video device (that's what our FrameBufferAllocator
> does, and I was testing with 'cam' which uses that class to reserve
> memory per-stream).
>
> I have a discussion open with Phil and Nicolas to see how those two
> patches might be made upstream consumable, but I'll remember to keep
> them in when testing.
OK, I'd seen some emails flying around about those patches, but then
couldn't find them when I went looking earlier.
> >
> > The latter is likely to be more involved and depend on what actually
> > allocated it (which may mean I'm using the wrong API calls).
> > I'm getting the dma address via sg_dma_address having attached and
> > mapped the dmabuf. I'm expecting that dma address to therefore follow
> > the dma-ranges of my device (not the allocator), but I'm not 100%
> > certain that is what does happen.
> >
> > A few more details of what exactly the use case that triggers it would
> > be useful.
> >
> > > >
> > > > Whilst waiting for that, the Unicam driver, and the prep work in
> > > > mmal-vchiq could all be pushed first, and ideally as two independent
> > > > patchsets as there are no inter-dependencies between them.
> > >
> > > I could start sending out the unicam driver, yes.
> > >
> > > Currently I'm a bit stuck not being able to receive frames from the
> > > unicam driver. I see the buffers being returned by the ISR routine,
> > > but I never get a buffer available notification in libcamera.
> > >
> > > There's been a few changes to the downstream unicam driver (ie
> > > requesting the VPU clock frequency) and I see the RPi mainline support
> > > has moved forward quite a bit since v5.8. Are you aware of any trivial
> > > change I might be missing that could cause this ?
> >
> > Things never stand still!
>
> Isn't it great ? :D
>
> > Requesting the VPU clock is to avoid a FIFO overflow.
> > We've gained pixel format support (with defines and docs) for Y12P and Y14P.
> > And hopefully I fixed up all the review comments from v1.
>
> Do you think it's worth upstreaming those parts in v2 or should it be
> done on top ? I should check if the VPU clock is exposed or not in
> mainline first...
Adding in the clock would be useful to avoid odd image issues
(particularly on imx477), but you're right it has a dependency on the
clock driver. I thought Maxime had managed to get that merged, but I
haven't checked.
Adding the image formats is fairly uncontentious as long as I've got
the docs for the image formats right. I did jump through the hoops to
test the docs, so I hope they're OK. The changes to the driver to
support them are a dozen lines adding them to a table.
> >
> > It's hard to guess why libcamera isn't happy if we're getting ISRs. We
> > do need to get the appropriate flags in the ISR for frame end (FEI or
> > PI0).
> > Is this Pi3 or Pi4 that you're testing on? Pi3 support should be sound
> > on mainline. AFAIK Pi4 is still a work in progress.
> > Can you stream just using v4l2-ctl --stream-mmap=3 --stream-count=1000
> > --stream-to=/dev/null ? That should work with /dev/video0. Getting the
> > embedded data is more involved over the ordering of opening and
> > starting streaming on the nodes.
>
> It's a Pi4.
Sorry, I've not tried mainline on a Pi4 yet.
> After a pleasant day of debugging I got a nice trace that shows me
> that it's actually the buffer importing part that hangs on a completion
> in vc_sm_cma_vchi_import.
>
> In mainline the whole vchi interface got dropped and I got to port the
> vc-sma-cma driver to use the vchiq interface directly. I might have
> screwed something up and I'm now looking suspiciously at this thread
> function: "vc_sm_cma_vchi_videocore_io()" (also because I had to move it
> to use msg_hold+msg_release, as peek+remove was killed by:
> b5f1547b6e3bd ("staging: vchi: Get rid of vchi_msg_peek()")
I'll look at those vchi/vchiq patches if they've now been merged. It
makes some sense for us to backport them so they get a real thrashing.
If you've gone through any of the existing drivers doing the
conversion, then feel free to throw me the files to have a look at
(and it potentially saves me some effort in doing the same
conversion).
> I'll look into that again.
>
> In the meantime is there any chance this rings any bell to you?
> https://paste.debian.net/1161469/
Nothing obvious - sorry.
> Thanks
> j
>
> >
> > (We do have an open issue regarding getting the correct DMA address
> > for Unicam on PI0 & 1. Those share the L2 cache for many things so
> > dma-ranges is set differently, and not in a way that currently works
> > with the Unicam driver. That one may need a followup fix.)
Issue resolved. My check in the driver was invalid - we have a
downstream PR for it now.
Dave
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-04 9:25 ` [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver Laurent Pinchart
` (3 preceding siblings ...)
2020-05-06 18:04 ` Nicolas Saenz Julienne
@ 2020-05-11 18:42 ` Nicolas Saenz Julienne
2020-05-18 15:48 ` Dave Stevenson
4 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-11 18:42 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 5368 bytes --]
Hi Dave,
some questions/comments.
On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> Add Broadcom VideoCore Shared Memory support.
>
> This new driver allows contiguous memory blocks to be imported
> into the VideoCore VPU memory map, and manages the lifetime of
> those objects, only releasing the source dmabuf once the VPU has
> confirmed it has finished with it.
>
> Driver upported from the RaspberryPi BSP at revision:
> 890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when counting
> transactions")
> forward ported to recent mainline kernel version.
>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
[...]
> +
> +/* Import a dma_buf to be shared with VC. */
> +int
> +vc_sm_cma_import_dmabuf_internal(struct vc_sm_privdata_t *private,
> + struct dma_buf *dma_buf,
> + int fd,
> + struct dma_buf **imported_buf)
> +{
> + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> + struct vc_sm_buffer *buffer = NULL;
> + struct vc_sm_import import = { };
> + struct vc_sm_import_result result = { };
> + struct dma_buf_attachment *attach = NULL;
> + struct sg_table *sgt = NULL;
> + dma_addr_t dma_addr;
> + int ret = 0;
> + int status;
> +
> + /* Setup our allocation parameters */
> + pr_debug("%s: importing dma_buf %p/fd %d\n", __func__, dma_buf, fd);
> +
> + if (fd < 0)
> + get_dma_buf(dma_buf);
> + else
> + dma_buf = dma_buf_get(fd);
> +
> + if (!dma_buf)
> + return -EINVAL;
> +
> + attach = dma_buf_attach(dma_buf, &sm_state->pdev->dev);
> + if (IS_ERR(attach)) {
> + ret = PTR_ERR(attach);
> + goto error;
> + }
> +
> + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> + if (IS_ERR(sgt)) {
> + ret = PTR_ERR(sgt);
> + goto error;
> + }
> +
> + /* Verify that the address block is contiguous */
> + if (sgt->nents != 1) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + /* Allocate local buffer to track this allocation. */
> + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> + if (!buffer) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + import.type = VC_SM_ALLOC_NON_CACHED;
> + dma_addr = sg_dma_address(sgt->sgl);
> + import.addr = (u32)dma_addr;
> + if ((import.addr & 0xC0000000) != 0xC0000000) {
> + pr_err("%s: Expecting an uncached alias for dma_addr %pad\n",
> + __func__, &dma_addr);
> + import.addr |= 0xC0000000;
> + }
Just so we don't forget about it, this shouldn't be needed once dma-ranges are
fixed.
> + import.size = sg_dma_len(sgt->sgl);
> + import.allocator = current->tgid;
> + import.kernel_id = get_kernel_id(buffer);
> +
> + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> +
> + pr_debug("[%s]: attempt to import \"%s\" data - type %u, addr %pad, size
> %u.\n",
> + __func__, import.name, import.type, &dma_addr, import.size);
> +
> + /* Allocate the videocore buffer. */
> + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import, &result,
> + &sm_state->int_trans_id);
> + if (status == -EINTR) {
> + pr_debug("[%s]: requesting import memory action restart
> (trans_id: %u)\n",
> + __func__, sm_state->int_trans_id);
> + ret = -ERESTARTSYS;
> + private->restart_sys = -EINTR;
> + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> + goto error;
> + } else if (status || !result.res_handle) {
> + pr_debug("[%s]: failed to import memory on videocore (status:
> %u, trans_id: %u)\n",
> + __func__, status, sm_state->int_trans_id);
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + mutex_init(&buffer->lock);
> + INIT_LIST_HEAD(&buffer->attachments);
> + memcpy(buffer->name, import.name,
> + min(sizeof(buffer->name), sizeof(import.name) - 1));
> +
> + /* Keep track of the buffer we created. */
> + buffer->private = private;
> + buffer->vc_handle = result.res_handle;
> + buffer->size = import.size;
> + buffer->vpu_state = VPU_MAPPED;
> +
> + buffer->imported = 1;
> + buffer->import.dma_buf = dma_buf;
> +
> + buffer->import.attach = attach;
> + buffer->import.sgt = sgt;
> + buffer->dma_addr = dma_addr;
> + buffer->in_use = 1;
> + buffer->kernel_id = import.kernel_id;
> +
> + /*
> + * We're done - we need to export a new dmabuf chaining through most
> + * functions, but enabling us to release our own internal references
> + * here.
> + */
> + exp_info.ops = &dma_buf_import_ops;
> + exp_info.size = import.size;
> + exp_info.flags = O_RDWR;
> + exp_info.priv = buffer;
> +
> + buffer->dma_buf = dma_buf_export(&exp_info);
Could you comment on the need for this second dma_buf? I've only reviewed code
related to mmal-vchiq imports, but it seems to me that it would be saner to do
the unmapping and unattaching explicitly as opposed to having this second
buffer refcount hit 0. Although, I can imagine this being needed for the
userspace interface.
When you talk about moving to dmabuf heaps, I've pictured a specific dmabuf
heap for vc4 that takes care of all the importing and unimporting (aside from
cma allocations). Am I right? If so, I'm pretty confident we can do away with
this.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-11 18:42 ` Nicolas Saenz Julienne
@ 2020-05-18 15:48 ` Dave Stevenson
2020-05-20 14:41 ` Nicolas Saenz Julienne
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-05-18 15:48 UTC (permalink / raw)
To: Nicolas Saenz Julienne
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck,
Dave Stevenson
Hi Nicolas
On Mon, 11 May 2020 at 19:43, Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Hi Dave,
> some questions/comments.
>
> On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> >
> > Add Broadcom VideoCore Shared Memory support.
> >
> > This new driver allows contiguous memory blocks to be imported
> > into the VideoCore VPU memory map, and manages the lifetime of
> > those objects, only releasing the source dmabuf once the VPU has
> > confirmed it has finished with it.
> >
> > Driver upported from the RaspberryPi BSP at revision:
> > 890691d1c996 ("staging: vc04_services: Fix vcsm overflow bug when counting
> > transactions")
> > forward ported to recent mainline kernel version.
> >
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
>
> [...]
>
> > +
> > +/* Import a dma_buf to be shared with VC. */
> > +int
> > +vc_sm_cma_import_dmabuf_internal(struct vc_sm_privdata_t *private,
> > + struct dma_buf *dma_buf,
> > + int fd,
> > + struct dma_buf **imported_buf)
> > +{
> > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
> > + struct vc_sm_buffer *buffer = NULL;
> > + struct vc_sm_import import = { };
> > + struct vc_sm_import_result result = { };
> > + struct dma_buf_attachment *attach = NULL;
> > + struct sg_table *sgt = NULL;
> > + dma_addr_t dma_addr;
> > + int ret = 0;
> > + int status;
> > +
> > + /* Setup our allocation parameters */
> > + pr_debug("%s: importing dma_buf %p/fd %d\n", __func__, dma_buf, fd);
> > +
> > + if (fd < 0)
> > + get_dma_buf(dma_buf);
> > + else
> > + dma_buf = dma_buf_get(fd);
> > +
> > + if (!dma_buf)
> > + return -EINVAL;
> > +
> > + attach = dma_buf_attach(dma_buf, &sm_state->pdev->dev);
> > + if (IS_ERR(attach)) {
> > + ret = PTR_ERR(attach);
> > + goto error;
> > + }
> > +
> > + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL);
> > + if (IS_ERR(sgt)) {
> > + ret = PTR_ERR(sgt);
> > + goto error;
> > + }
> > +
> > + /* Verify that the address block is contiguous */
> > + if (sgt->nents != 1) {
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + /* Allocate local buffer to track this allocation. */
> > + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL);
> > + if (!buffer) {
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + import.type = VC_SM_ALLOC_NON_CACHED;
> > + dma_addr = sg_dma_address(sgt->sgl);
> > + import.addr = (u32)dma_addr;
> > + if ((import.addr & 0xC0000000) != 0xC0000000) {
> > + pr_err("%s: Expecting an uncached alias for dma_addr %pad\n",
> > + __func__, &dma_addr);
> > + import.addr |= 0xC0000000;
> > + }
>
> Just so we don't forget about it, this shouldn't be needed once dma-ranges are
> fixed.
Indeed not, but we've had enough issues with dma-ranges going missing
that screaming and shouting about it is a good thing.
> > + import.size = sg_dma_len(sgt->sgl);
> > + import.allocator = current->tgid;
> > + import.kernel_id = get_kernel_id(buffer);
> > +
> > + memcpy(import.name, VC_SM_RESOURCE_NAME_DEFAULT,
> > + sizeof(VC_SM_RESOURCE_NAME_DEFAULT));
> > +
> > + pr_debug("[%s]: attempt to import \"%s\" data - type %u, addr %pad, size
> > %u.\n",
> > + __func__, import.name, import.type, &dma_addr, import.size);
> > +
> > + /* Allocate the videocore buffer. */
> > + status = vc_sm_cma_vchi_import(sm_state->sm_handle, &import, &result,
> > + &sm_state->int_trans_id);
> > + if (status == -EINTR) {
> > + pr_debug("[%s]: requesting import memory action restart
> > (trans_id: %u)\n",
> > + __func__, sm_state->int_trans_id);
> > + ret = -ERESTARTSYS;
> > + private->restart_sys = -EINTR;
> > + private->int_action = VC_SM_MSG_TYPE_IMPORT;
> > + goto error;
> > + } else if (status || !result.res_handle) {
> > + pr_debug("[%s]: failed to import memory on videocore (status:
> > %u, trans_id: %u)\n",
> > + __func__, status, sm_state->int_trans_id);
> > + ret = -ENOMEM;
> > + goto error;
> > + }
> > +
> > + mutex_init(&buffer->lock);
> > + INIT_LIST_HEAD(&buffer->attachments);
> > + memcpy(buffer->name, import.name,
> > + min(sizeof(buffer->name), sizeof(import.name) - 1));
> > +
> > + /* Keep track of the buffer we created. */
> > + buffer->private = private;
> > + buffer->vc_handle = result.res_handle;
> > + buffer->size = import.size;
> > + buffer->vpu_state = VPU_MAPPED;
> > +
> > + buffer->imported = 1;
> > + buffer->import.dma_buf = dma_buf;
> > +
> > + buffer->import.attach = attach;
> > + buffer->import.sgt = sgt;
> > + buffer->dma_addr = dma_addr;
> > + buffer->in_use = 1;
> > + buffer->kernel_id = import.kernel_id;
> > +
> > + /*
> > + * We're done - we need to export a new dmabuf chaining through most
> > + * functions, but enabling us to release our own internal references
> > + * here.
> > + */
> > + exp_info.ops = &dma_buf_import_ops;
> > + exp_info.size = import.size;
> > + exp_info.flags = O_RDWR;
> > + exp_info.priv = buffer;
> > +
> > + buffer->dma_buf = dma_buf_export(&exp_info);
>
> Could you comment on the need for this second dma_buf? I've only reviewed code
> related to mmal-vchiq imports, but it seems to me that it would be saner to do
> the unmapping and unattaching explicitly as opposed to having this second
> buffer refcount hit 0. Although, I can imagine this being needed for the
> userspace interface.
Indeed, as it is needed for the userspace interface it seemed to make
more sense to have common handling rather than two code paths doing
nearly the same thing but in different ways.
Downstream we need a userspace import at least to allow MMAL to set up
zero copy, so unless it raises any real objections then it would be
useful to keep it.
> When you talk about moving to dmabuf heaps, I've pictured a specific dmabuf
> heap for vc4 that takes care of all the importing and unimporting (aside from
> cma allocations). Am I right? If so, I'm pretty confident we can do away with
> this.
(Note I'm talking about the VideoCore4 VPU and other blocks, and not
the vc4 DRM/KMS and V3D drivers)
No, I'm looking at using the existing cma_heap driver to do the
allocations, and then this driver will import them and handle the
lifetime on behalf of the VPU. There's no need for VPU allocations to
be split off into yet another heap.
One of the things we are trying to get away from is having the gpu_mem
reserved lump that Linux can't get access to at all, so allocating
from the CMA heap and importing to the VPU avoids that.
I'll give some history here, which also hopefully covers your query
over switching mmal-vchiq to zero copy.
Almost all the VC4 blocks need contiguous memory, so fragmentation was
an issue. To resolve that we (back in Broadcom days) had the
"relocatable heap" - allocations that needed to be locked before
access and unlocked after. Unlocked blocks could be copied and moved
around to free up larger contiguous blocks. These allocations use a
handle instead of a pointer, and have internal refcounting etc.
Basically providing some of the features of an MMU when you don't have
one.
The original VCSM driver allowed userspace to make a relocatable heap
allocation, lock it, and the kernel to map the relevant pages into the
ARM memory space. Now you have a shared buffer, and VCHIQ no longer
has to copy the data back and forth. (Cache flushing was also
handled).
So MMAL in zero copy mode passes the VPU relocatable heap handle
across in the VCHIQ message, not a pointer to the actual data. VCSM
did the allocation on behalf of the MMAL client, and provides the
mapping and VPU handle to the buffer. This still leaves the allocation
being made from gpu_mem though.
The rewrite (vc-sm-cma) was to make use of an import feature into the
relocatable heap, termed internally as mem wrapping. Take a CMA
allocation made by something, pass the DMA address and size across to
the VPU, and it can insert it as a relocatable heap object that can be
used in exactly the same way gpu_mem allocations. gpu_mem can now be
shrunk in size :-) It was using a dma-buf as a convenient object to
manage the allocation, and handle importing buffers allocated by other
subsystems
Note that we still have refcounting internally to the relocatable
heap, so at the point the client says it has finished with it, the VPU
may not have done. When the last relocatable heap reference is
released, the kernel gets a callback (VC_SM_MSG_TYPE_RELEASED), and it
is only at that point that it is safe to drop the reference to the
imported dmabuf.
V4L2 can do the relevant import and wrapping to a relocatable heap
handle as part of the buffer passing. MMAL needs to do it manually
from userspace as VCHIQ is the only in-kernel service that it uses,
hence we need an import ioctl and free mechanism (if the handle is a
dmabuf, then that's close).
From a platform level it would be nice to have the userspace ioctl for
importing a dmabuf in mainline, however it isn't necessary for the
V4L2 use cases that we're trying to upstream here. The driver without
userspace API would look pretty much like the one in [1]. I'll try and
update that to include the basic import userspace API to give a
comparison.
I don't mind which way this goes as to whether the userspace ioctl
remains as downstream patches, but losing the dmabuf as the handle
within vc-sm-cma will make that patch huge, and they're almost
guaranteed to diverge.
Ignore the caching ioctls - they're irrelevant.
I hope that makes the situation a little clearer.
Dave
[1] https://github.com/6by9/linux/tree/staging_next_upstreaming_apr20/drivers/staging/vc04_services/vc-sm-cma
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-18 15:48 ` Dave Stevenson
@ 2020-05-20 14:41 ` Nicolas Saenz Julienne
2020-05-21 11:01 ` Dave Stevenson
0 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-20 14:41 UTC (permalink / raw)
To: Dave Stevenson
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck,
Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 5428 bytes --]
On Mon, 2020-05-18 at 16:48 +0100, Dave Stevenson wrote:
> Hi Nicolas
> > > + exp_info.ops = &dma_buf_import_ops;
> > > + exp_info.size = import.size;
> > > + exp_info.flags = O_RDWR;
> > > + exp_info.priv = buffer;
> > > +
> > > + buffer->dma_buf = dma_buf_export(&exp_info);
> >
> > Could you comment on the need for this second dma_buf? I've only reviewed
> > code
> > related to mmal-vchiq imports, but it seems to me that it would be saner to
> > do
> > the unmapping and unattaching explicitly as opposed to having this second
> > buffer refcount hit 0. Although, I can imagine this being needed for the
> > userspace interface.
>
> Indeed, as it is needed for the userspace interface it seemed to make
> more sense to have common handling rather than two code paths doing
> nearly the same thing but in different ways.
> Downstream we need a userspace import at least to allow MMAL to set up
> zero copy, so unless it raises any real objections then it would be
> useful to keep it.
>
> > When you talk about moving to dmabuf heaps, I've pictured a specific dmabuf
> > heap for vc4 that takes care of all the importing and unimporting (aside
> > from
> > cma allocations). Am I right? If so, I'm pretty confident we can do away
> > with
> > this.
>
> (Note I'm talking about the VideoCore4 VPU and other blocks, and not
> the vc4 DRM/KMS and V3D drivers)
>
> No, I'm looking at using the existing cma_heap driver to do the
> allocations, and then this driver will import them and handle the
> lifetime on behalf of the VPU. There's no need for VPU allocations to
> be split off into yet another heap.
Fair enough.
> One of the things we are trying to get away from is having the gpu_mem
> reserved lump that Linux can't get access to at all, so allocating
> from the CMA heap and importing to the VPU avoids that.
That's great! Will this also apply at some point to the GPU side of things? I
vaguely recall having to reserve up to 300M on rpi3 to get mesa to work on a
desktop environment.
> I'll give some history here, which also hopefully covers your query
> over switching mmal-vchiq to zero copy.
>
> Almost all the VC4 blocks need contiguous memory, so fragmentation was
> an issue. To resolve that we (back in Broadcom days) had the
> "relocatable heap" - allocations that needed to be locked before
> access and unlocked after. Unlocked blocks could be copied and moved
> around to free up larger contiguous blocks. These allocations use a
> handle instead of a pointer, and have internal refcounting etc.
> Basically providing some of the features of an MMU when you don't have
> one.
>
> The original VCSM driver allowed userspace to make a relocatable heap
> allocation, lock it, and the kernel to map the relevant pages into the
> ARM memory space. Now you have a shared buffer, and VCHIQ no longer
> has to copy the data back and forth. (Cache flushing was also
> handled).
> So MMAL in zero copy mode passes the VPU relocatable heap handle
> across in the VCHIQ message, not a pointer to the actual data. VCSM
> did the allocation on behalf of the MMAL client, and provides the
> mapping and VPU handle to the buffer. This still leaves the allocation
> being made from gpu_mem though.
>
> The rewrite (vc-sm-cma) was to make use of an import feature into the
> relocatable heap, termed internally as mem wrapping. Take a CMA
> allocation made by something, pass the DMA address and size across to
> the VPU, and it can insert it as a relocatable heap object that can be
> used in exactly the same way gpu_mem allocations. gpu_mem can now be
> shrunk in size :-) It was using a dma-buf as a convenient object to
> manage the allocation, and handle importing buffers allocated by other
> subsystems
> Note that we still have refcounting internally to the relocatable
> heap, so at the point the client says it has finished with it, the VPU
> may not have done. When the last relocatable heap reference is
> released, the kernel gets a callback (VC_SM_MSG_TYPE_RELEASED), and it
> is only at that point that it is safe to drop the reference to the
> imported dmabuf.
>
> V4L2 can do the relevant import and wrapping to a relocatable heap
> handle as part of the buffer passing. MMAL needs to do it manually
> from userspace as VCHIQ is the only in-kernel service that it uses,
> hence we need an import ioctl and free mechanism (if the handle is a
> dmabuf, then that's close).
>
>
> From a platform level it would be nice to have the userspace ioctl for
> importing a dmabuf in mainline, however it isn't necessary for the
> V4L2 use cases that we're trying to upstream here. The driver without
> userspace API would look pretty much like the one in [1]. I'll try and
> update that to include the basic import userspace API to give a
> comparison.
> I don't mind which way this goes as to whether the userspace ioctl
> remains as downstream patches, but losing the dmabuf as the handle
> within vc-sm-cma will make that patch huge, and they're almost
> guaranteed to diverge.
> Ignore the caching ioctls - they're irrelevant.
>
> I hope that makes the situation a little clearer.
Thanks a lot for the in-depth explanation, it does indeed. Actually, it
wouldn't hurt to add a subset of this into a text file alongside with the new
driver.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver
2020-05-20 14:41 ` Nicolas Saenz Julienne
@ 2020-05-21 11:01 ` Dave Stevenson
0 siblings, 0 replies; 111+ messages in thread
From: Dave Stevenson @ 2020-05-21 11:01 UTC (permalink / raw)
To: Nicolas Saenz Julienne
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck,
Dave Stevenson
Hi Nicholas
On Wed, 20 May 2020 at 15:41, Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> On Mon, 2020-05-18 at 16:48 +0100, Dave Stevenson wrote:
> > Hi Nicolas
> > > > + exp_info.ops = &dma_buf_import_ops;
> > > > + exp_info.size = import.size;
> > > > + exp_info.flags = O_RDWR;
> > > > + exp_info.priv = buffer;
> > > > +
> > > > + buffer->dma_buf = dma_buf_export(&exp_info);
> > >
> > > Could you comment on the need for this second dma_buf? I've only reviewed
> > > code
> > > related to mmal-vchiq imports, but it seems to me that it would be saner to
> > > do
> > > the unmapping and unattaching explicitly as opposed to having this second
> > > buffer refcount hit 0. Although, I can imagine this being needed for the
> > > userspace interface.
> >
> > Indeed, as it is needed for the userspace interface it seemed to make
> > more sense to have common handling rather than two code paths doing
> > nearly the same thing but in different ways.
> > Downstream we need a userspace import at least to allow MMAL to set up
> > zero copy, so unless it raises any real objections then it would be
> > useful to keep it.
> >
> > > When you talk about moving to dmabuf heaps, I've pictured a specific dmabuf
> > > heap for vc4 that takes care of all the importing and unimporting (aside
> > > from
> > > cma allocations). Am I right? If so, I'm pretty confident we can do away
> > > with
> > > this.
> >
> > (Note I'm talking about the VideoCore4 VPU and other blocks, and not
> > the vc4 DRM/KMS and V3D drivers)
> >
> > No, I'm looking at using the existing cma_heap driver to do the
> > allocations, and then this driver will import them and handle the
> > lifetime on behalf of the VPU. There's no need for VPU allocations to
> > be split off into yet another heap.
>
> Fair enough.
>
> > One of the things we are trying to get away from is having the gpu_mem
> > reserved lump that Linux can't get access to at all, so allocating
> > from the CMA heap and importing to the VPU avoids that.
>
> That's great! Will this also apply at some point to the GPU side of things? I
> vaguely recall having to reserve up to 300M on rpi3 to get mesa to work on a
> desktop environment.
None of this has any involvement with the vc4 V3D or v3d DRM/KMS
drivers. gpu_mem (the size of the VPU's relocatable heap) shouldn't
have affected that at all other than being another carveout.
I did have code working where VPU relocatable heap alloc calls made a
VCHIQ call across to this driver, it made an allocation using
dma_alloc_contig, and then passed all the required information back.
Whilst it worked in all the use cases we tested, there were still a
few reservations over potential deadlocks depending on exactly who
called what when. There were also discussions on how to tell users
that everything had changed, gpu_mem no longer did anything much, and
please set up CMA regions. The best time is probably on either a new
Debian/Raspbian release, or possibly when bumping to the next LTS
kernel, but we just know there will be complaints whenever we do it.
If/when the world quietens down a bit I may look at resurrecting it
again. It fitted in better when this driver was doing allocations, but
if we can arrange that there is no need to work backwards from an
internal allocation VPU handle to a dmabuf, then actually the
allocation and dma map code can be relatively self-contained.
> > I'll give some history here, which also hopefully covers your query
> > over switching mmal-vchiq to zero copy.
> >
> > Almost all the VC4 blocks need contiguous memory, so fragmentation was
> > an issue. To resolve that we (back in Broadcom days) had the
> > "relocatable heap" - allocations that needed to be locked before
> > access and unlocked after. Unlocked blocks could be copied and moved
> > around to free up larger contiguous blocks. These allocations use a
> > handle instead of a pointer, and have internal refcounting etc.
> > Basically providing some of the features of an MMU when you don't have
> > one.
> >
> > The original VCSM driver allowed userspace to make a relocatable heap
> > allocation, lock it, and the kernel to map the relevant pages into the
> > ARM memory space. Now you have a shared buffer, and VCHIQ no longer
> > has to copy the data back and forth. (Cache flushing was also
> > handled).
> > So MMAL in zero copy mode passes the VPU relocatable heap handle
> > across in the VCHIQ message, not a pointer to the actual data. VCSM
> > did the allocation on behalf of the MMAL client, and provides the
> > mapping and VPU handle to the buffer. This still leaves the allocation
> > being made from gpu_mem though.
> >
> > The rewrite (vc-sm-cma) was to make use of an import feature into the
> > relocatable heap, termed internally as mem wrapping. Take a CMA
> > allocation made by something, pass the DMA address and size across to
> > the VPU, and it can insert it as a relocatable heap object that can be
> > used in exactly the same way gpu_mem allocations. gpu_mem can now be
> > shrunk in size :-) It was using a dma-buf as a convenient object to
> > manage the allocation, and handle importing buffers allocated by other
> > subsystems
> > Note that we still have refcounting internally to the relocatable
> > heap, so at the point the client says it has finished with it, the VPU
> > may not have done. When the last relocatable heap reference is
> > released, the kernel gets a callback (VC_SM_MSG_TYPE_RELEASED), and it
> > is only at that point that it is safe to drop the reference to the
> > imported dmabuf.
> >
> > V4L2 can do the relevant import and wrapping to a relocatable heap
> > handle as part of the buffer passing. MMAL needs to do it manually
> > from userspace as VCHIQ is the only in-kernel service that it uses,
> > hence we need an import ioctl and free mechanism (if the handle is a
> > dmabuf, then that's close).
> >
> >
> > From a platform level it would be nice to have the userspace ioctl for
> > importing a dmabuf in mainline, however it isn't necessary for the
> > V4L2 use cases that we're trying to upstream here. The driver without
> > userspace API would look pretty much like the one in [1]. I'll try and
> > update that to include the basic import userspace API to give a
> > comparison.
> > I don't mind which way this goes as to whether the userspace ioctl
> > remains as downstream patches, but losing the dmabuf as the handle
> > within vc-sm-cma will make that patch huge, and they're almost
> > guaranteed to diverge.
> > Ignore the caching ioctls - they're irrelevant.
> >
> > I hope that makes the situation a little clearer.
>
> Thanks a lot for the in-depth explanation, it does indeed. Actually, it
> wouldn't hurt to add a subset of this into a text file alongside with the new
> driver.
OK, I'll look at tidying things up before Laurent and co push another
version for review.
Dave
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 07/34] staging: bcm2835: Break MMAL support out from camera
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (5 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 06/34] staging: vc04_services: Add new vc-sm-cma driver Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 08/34] staging: mmal-vchiq: Allocate and free components as required Laurent Pinchart
` (27 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Jacopo Mondi <jacopo@jmondi.org>
The BCM2835 camera host is currently the only component that uses the
VCHIQ MMAL interface. This will soon change with the upporting of
BCM2835 ISP, which make use of the same interface.
Break VCHIQ MMAL interface support out from camera host directory to
make it possible for the ISP driver to use it as well.
The only modification to the existing mmal code is the introduction of
EXPORT_SYMBOL() for symbols required by bcm2835-camera and the addition
of the module author and licenses.
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/Kconfig | 2 ++
drivers/staging/vc04_services/Makefile | 7 +++---
.../vc04_services/bcm2835-camera/Kconfig | 1 +
.../vc04_services/bcm2835-camera/Makefile | 4 ++--
.../staging/vc04_services/vchiq-mmal/Kconfig | 8 +++++++
.../staging/vc04_services/vchiq-mmal/Makefile | 8 +++++++
.../mmal-common.h | 0
.../mmal-encodings.h | 0
.../mmal-msg-common.h | 0
.../mmal-msg-format.h | 0
.../mmal-msg-port.h | 0
.../{bcm2835-camera => vchiq-mmal}/mmal-msg.h | 0
.../mmal-parameters.h | 0
.../mmal-vchiq.c | 22 +++++++++++++++++++
.../mmal-vchiq.h | 0
15 files changed, 47 insertions(+), 5 deletions(-)
create mode 100644 drivers/staging/vc04_services/vchiq-mmal/Kconfig
create mode 100644 drivers/staging/vc04_services/vchiq-mmal/Makefile
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-common.h (100%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-encodings.h (100%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-msg-common.h (100%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-msg-format.h (100%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-msg-port.h (100%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-msg.h (100%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-parameters.h (100%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-vchiq.c (98%)
rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-vchiq.h (100%)
diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
index b9f1f019ebd8..7ae21d7f359b 100644
--- a/drivers/staging/vc04_services/Kconfig
+++ b/drivers/staging/vc04_services/Kconfig
@@ -25,5 +25,7 @@ source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
+source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
+
endif
diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
index 6e1abf494c1a..296aff05b9ab 100644
--- a/drivers/staging/vc04_services/Makefile
+++ b/drivers/staging/vc04_services/Makefile
@@ -10,9 +10,10 @@ vchiq-objs := \
interface/vchiq_arm/vchiq_util.o \
interface/vchiq_arm/vchiq_connected.o \
-obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
-obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
-obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
+obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
+obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
+obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
+obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
ccflags-y += -D__VCCOREVER__=0x04000000
diff --git a/drivers/staging/vc04_services/bcm2835-camera/Kconfig b/drivers/staging/vc04_services/bcm2835-camera/Kconfig
index c81baf2c111e..d0653d1ed3c7 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/Kconfig
+++ b/drivers/staging/vc04_services/bcm2835-camera/Kconfig
@@ -4,6 +4,7 @@ config VIDEO_BCM2835
depends on MEDIA_SUPPORT
depends on VIDEO_V4L2 && (ARCH_BCM2835 || COMPILE_TEST)
select BCM2835_VCHIQ
+ select BCM2835_VCHIQ_MMAL
select VIDEOBUF2_VMALLOC
select BTREE
help
diff --git a/drivers/staging/vc04_services/bcm2835-camera/Makefile b/drivers/staging/vc04_services/bcm2835-camera/Makefile
index 472f21e1f2a1..3a76d6ade428 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/Makefile
+++ b/drivers/staging/vc04_services/bcm2835-camera/Makefile
@@ -1,11 +1,11 @@
# SPDX-License-Identifier: GPL-2.0
bcm2835-v4l2-$(CONFIG_VIDEO_BCM2835) := \
bcm2835-camera.o \
- controls.o \
- mmal-vchiq.o
+ controls.o
obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-v4l2.o
ccflags-y += \
-I $(srctree)/$(src)/.. \
+ -I $(srctree)/$(src)/../vchiq-mmal/ \
-D__VCCOREVER__=0x04000000
diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
new file mode 100644
index 000000000000..106f71e709df
--- /dev/null
+++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
@@ -0,0 +1,8 @@
+config BCM2835_VCHIQ_MMAL
+ tristate "BCM2835 MMAL VCHIQ service"
+ depends on (ARCH_BCM2835 || COMPILE_TEST)
+ select BCM_VC_SM_CMA
+ help
+ Enables the MMAL API over VCHIQ interface as used for the
+ majority of the multimedia services on VideoCore.
+ Defaults to Y when the Broadcomd BCM2835 camera host is selected.
diff --git a/drivers/staging/vc04_services/vchiq-mmal/Makefile b/drivers/staging/vc04_services/vchiq-mmal/Makefile
new file mode 100644
index 000000000000..f8164c33aec3
--- /dev/null
+++ b/drivers/staging/vc04_services/vchiq-mmal/Makefile
@@ -0,0 +1,8 @@
+# SPDX-License-Identifier: GPL-2.0
+bcm2835-mmal-vchiq-objs := mmal-vchiq.o
+
+obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += bcm2835-mmal-vchiq.o
+
+ccflags-y += \
+ -I$(srctree)/$(src)/.. \
+ -D__VCCOREVER__=0x04000000
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-common.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-common.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-encodings.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-msg-common.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg-common.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-msg-common.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-msg-common.h
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-msg-format.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg-format.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-msg-format.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-msg-format.h
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-msg-port.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg-port.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-msg-port.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-msg-port.h
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-msg.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-msg.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-parameters.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
similarity index 98%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-vchiq.c
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index de03b90021a8..cb3ffebb9ace 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -17,6 +17,7 @@
#include <linux/errno.h>
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/mm.h>
#include <linux/slab.h>
@@ -1396,6 +1397,7 @@ int vchiq_mmal_port_set_format(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_port_set_format);
int vchiq_mmal_port_parameter_set(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_port *port,
@@ -1412,6 +1414,7 @@ int vchiq_mmal_port_parameter_set(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_port_parameter_set);
int vchiq_mmal_port_parameter_get(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_port *port,
@@ -1428,6 +1431,7 @@ int vchiq_mmal_port_parameter_get(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_port_parameter_get);
/* enable a port
*
@@ -1458,6 +1462,7 @@ int vchiq_mmal_port_enable(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_port_enable);
int vchiq_mmal_port_disable(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_port *port)
@@ -1478,6 +1483,7 @@ int vchiq_mmal_port_disable(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_port_disable);
/* ports will be connected in a tunneled manner so data buffers
* are not handled by client.
@@ -1565,6 +1571,7 @@ int vchiq_mmal_port_connect_tunnel(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_port_connect_tunnel);
int vchiq_mmal_submit_buffer(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_port *port,
@@ -1583,6 +1590,7 @@ int vchiq_mmal_submit_buffer(struct vchiq_mmal_instance *instance,
return 0;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_submit_buffer);
int mmal_vchi_buffer_init(struct vchiq_mmal_instance *instance,
struct mmal_buffer *buf)
@@ -1595,6 +1603,7 @@ int mmal_vchi_buffer_init(struct vchiq_mmal_instance *instance,
buf->msg_context = msg_context;
return 0;
}
+EXPORT_SYMBOL_GPL(mmal_vchi_buffer_init);
int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
{
@@ -1606,6 +1615,7 @@ int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
return 0;
}
+EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
/* Initialise a mmal component and its ports
*
@@ -1693,6 +1703,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_component_init);
/*
* cause a mmal component to be destroyed
@@ -1714,6 +1725,7 @@ int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_component_finalise);
/*
* cause a mmal component to be enabled
@@ -1739,6 +1751,7 @@ int vchiq_mmal_component_enable(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_component_enable);
/*
* cause a mmal component to be enabled
@@ -1764,6 +1777,7 @@ int vchiq_mmal_component_disable(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_component_disable);
int vchiq_mmal_version(struct vchiq_mmal_instance *instance,
u32 *major_out, u32 *minor_out)
@@ -1779,6 +1793,7 @@ int vchiq_mmal_version(struct vchiq_mmal_instance *instance,
return ret;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_version);
int vchiq_mmal_finalise(struct vchiq_mmal_instance *instance)
{
@@ -1809,6 +1824,7 @@ int vchiq_mmal_finalise(struct vchiq_mmal_instance *instance)
return status;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_finalise);
int vchiq_mmal_init(struct vchiq_mmal_instance **out_instance)
{
@@ -1889,3 +1905,9 @@ int vchiq_mmal_init(struct vchiq_mmal_instance **out_instance)
kfree(instance);
return -ENODEV;
}
+EXPORT_SYMBOL_GPL(vchiq_mmal_init);
+
+MODULE_DESCRIPTION("BCM2835 MMAL VCHIQ interface");
+MODULE_AUTHOR("Dave Stevenson, <dave.stevenson@raspberrypi.org>");
+MODULE_LICENSE("GPL");
+MODULE_VERSION("0.0.1");
diff --git a/drivers/staging/vc04_services/bcm2835-camera/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
similarity index 100%
rename from drivers/staging/vc04_services/bcm2835-camera/mmal-vchiq.h
rename to drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 08/34] staging: mmal-vchiq: Allocate and free components as required
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (6 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 07/34] staging: bcm2835: Break MMAL support out from camera Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 09/34] staging: mmal-vchiq: Avoid use of bool in structures Laurent Pinchart
` (26 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
The existing code assumed that there would only ever be 4 components,
and never freed the entries once used.
Allow arbitrary creation and destruction of components.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 29 ++++++++++++-------
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
2 files changed, 20 insertions(+), 10 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index cb3ffebb9ace..cf38ec78f77f 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -32,8 +32,11 @@
#define USE_VCHIQ_ARM
#include "interface/vchi/vchi.h"
-/* maximum number of components supported */
-#define VCHIQ_MMAL_MAX_COMPONENTS 4
+/*
+ * maximum number of components supported.
+ * This matches the maximum permitted by default on the VPU
+ */
+#define VCHIQ_MMAL_MAX_COMPONENTS 64
/*#define FULL_MSG_DUMP 1*/
@@ -168,8 +171,6 @@ struct vchiq_mmal_instance {
/* protect accesses to context_map */
struct mutex context_map_lock;
- /* component to use next */
- int component_idx;
struct vchiq_mmal_component component[VCHIQ_MMAL_MAX_COMPONENTS];
/* ordered workqueue to process all bulk operations */
@@ -1626,18 +1627,24 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
{
int ret;
int idx; /* port index */
- struct vchiq_mmal_component *component;
+ struct vchiq_mmal_component *component = NULL;
if (mutex_lock_interruptible(&instance->vchiq_mutex))
return -EINTR;
- if (instance->component_idx == VCHIQ_MMAL_MAX_COMPONENTS) {
+ for (idx = 0; idx < VCHIQ_MMAL_MAX_COMPONENTS; idx++) {
+ if (!instance->component[idx].in_use) {
+ component = &instance->component[idx];
+ component->in_use = 1;
+ break;
+ }
+ }
+
+ if (!component) {
ret = -EINVAL; /* todo is this correct error? */
goto unlock;
}
- component = &instance->component[instance->component_idx];
-
ret = create_component(instance, component, name);
if (ret < 0) {
pr_err("%s: failed to create component %d (Not enough GPU mem?)\n",
@@ -1688,8 +1695,6 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
goto release_component;
}
- instance->component_idx++;
-
*component_out = component;
mutex_unlock(&instance->vchiq_mutex);
@@ -1699,6 +1704,8 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
release_component:
destroy_component(instance, component);
unlock:
+ if (component)
+ component->in_use = 0;
mutex_unlock(&instance->vchiq_mutex);
return ret;
@@ -1721,6 +1728,8 @@ int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
ret = destroy_component(instance, component);
+ component->in_use = 0;
+
mutex_unlock(&instance->vchiq_mutex);
return ret;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index 47897e81ec58..4e34728d87e5 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -82,6 +82,7 @@ struct vchiq_mmal_port {
};
struct vchiq_mmal_component {
+ u32 in_use:1;
u32 enabled:1;
u32 handle; /* VideoCore handle for component */
u32 inputs; /* Number of input ports */
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 09/34] staging: mmal-vchiq: Avoid use of bool in structures
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (7 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 08/34] staging: mmal-vchiq: Allocate and free components as required Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 10/34] staging: mmal-vchiq: Make timeout a defined parameter Laurent Pinchart
` (25 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
Fixes up a checkpatch error "Avoid using bool structure members
because of possible alignment issues".
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index cf38ec78f77f..1ac27755df51 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -1754,7 +1754,7 @@ int vchiq_mmal_component_enable(struct vchiq_mmal_instance *instance,
ret = enable_component(instance, component);
if (ret == 0)
- component->enabled = true;
+ component->enabled = 1;
mutex_unlock(&instance->vchiq_mutex);
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 10/34] staging: mmal-vchiq: Make timeout a defined parameter
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (8 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 09/34] staging: mmal-vchiq: Avoid use of bool in structures Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 11/34] staging: mmal-vchiq: Make a mmal_buf struct for passing parameters Laurent Pinchart
` (24 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
The timeout period for VPU communications is a useful thing
to extend when debugging.
Set it via a define, rather than a magic number buried in the code.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 1ac27755df51..12c2f86b02ad 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -38,6 +38,12 @@
*/
#define VCHIQ_MMAL_MAX_COMPONENTS 64
+/*
+ * Timeout for synchronous msg responses in seconds.
+ * Helpful to increase this if stopping in the VPU debugger.
+ */
+#define SYNC_MSG_TIMEOUT 3
+
/*#define FULL_MSG_DUMP 1*/
#ifdef DEBUG
@@ -686,7 +692,7 @@ static int send_synchronous_mmal_msg(struct vchiq_mmal_instance *instance,
}
timeout = wait_for_completion_timeout(&msg_context->u.sync.cmplt,
- 3 * HZ);
+ SYNC_MSG_TIMEOUT * HZ);
if (timeout == 0) {
pr_err("timed out waiting for sync completion\n");
ret = -ETIME;
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 11/34] staging: mmal-vchiq: Make a mmal_buf struct for passing parameters
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (9 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 10/34] staging: mmal-vchiq: Make timeout a defined parameter Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 12/34] staging: mmal-vchiq: Add support for event callbacks Laurent Pinchart
` (23 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
The callback from vchi_mmal to the client was growing lots of extra
parameters. Consolidate them into a single struct instead of
growing the list further.
The struct is associated with the client buffer, therefore there
are various changes to setup various containers for the struct,
and pass the appropriate members.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../bcm2835-camera/bcm2835-camera.c | 62 ++++++++++++-------
.../vc04_services/vchiq-mmal/mmal-common.h | 5 ++
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 29 ++++++---
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 3 +-
4 files changed, 64 insertions(+), 35 deletions(-)
diff --git a/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c b/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
index 597acef35d0b..68a2260d88f5 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
+++ b/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
@@ -75,6 +75,12 @@ static const struct v4l2_fract
tpf_max = {.numerator = 1, .denominator = FPS_MIN},
tpf_default = {.numerator = 1000, .denominator = 30000};
+/* Container for MMAL and VB2 buffers*/
+struct vb2_mmal_buffer {
+ struct vb2_v4l2_buffer vb;
+ struct mmal_buffer mmal;
+};
+
/* video formats */
static struct mmal_fmt formats[] = {
{
@@ -261,14 +267,15 @@ static int buffer_init(struct vb2_buffer *vb)
{
struct bm2835_mmal_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
- struct mmal_buffer *buf = container_of(vb2, struct mmal_buffer, vb);
+ struct vb2_mmal_buffer *buf =
+ container_of(vb2, struct vb2_mmal_buffer, vb);
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev, "%s: dev:%p, vb %p\n",
__func__, dev, vb);
- buf->buffer = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
- buf->buffer_size = vb2_plane_size(&buf->vb.vb2_buf, 0);
+ buf->mmal.buffer = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
+ buf->mmal.buffer_size = vb2_plane_size(&buf->vb.vb2_buf, 0);
- return mmal_vchi_buffer_init(dev->instance, buf);
+ return mmal_vchi_buffer_init(dev->instance, &buf->mmal);
}
static int buffer_prepare(struct vb2_buffer *vb)
@@ -297,11 +304,13 @@ static void buffer_cleanup(struct vb2_buffer *vb)
{
struct bm2835_mmal_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
- struct mmal_buffer *buf = container_of(vb2, struct mmal_buffer, vb);
+ struct vb2_mmal_buffer *buf =
+ container_of(vb2, struct vb2_mmal_buffer, vb);
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev, "%s: dev:%p, vb %p\n",
__func__, dev, vb);
- mmal_vchi_buffer_cleanup(buf);
+
+ mmal_vchi_buffer_cleanup(&buf->mmal);
}
static inline bool is_capturing(struct bm2835_mmal_dev *dev)
@@ -313,14 +322,16 @@ static inline bool is_capturing(struct bm2835_mmal_dev *dev)
static void buffer_cb(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_port *port,
int status,
- struct mmal_buffer *buf,
- unsigned long length, u32 mmal_flags, s64 dts, s64 pts)
+ struct mmal_buffer *mmal_buf)
{
struct bm2835_mmal_dev *dev = port->cb_ctx;
+ struct vb2_mmal_buffer *buf =
+ container_of(mmal_buf, struct vb2_mmal_buffer, mmal);
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev,
"%s: status:%d, buf:%p, length:%lu, flags %u, pts %lld\n",
- __func__, status, buf, length, mmal_flags, pts);
+ __func__, status, buf, mmal_buf->length, mmal_buf->mmal_flags,
+ mmal_buf->pts);
if (status) {
/* error in transfer */
@@ -331,7 +342,7 @@ static void buffer_cb(struct vchiq_mmal_instance *instance,
return;
}
- if (length == 0) {
+ if (mmal_buf->length == 0) {
/* stream ended */
if (dev->capture.frame_count) {
/* empty buffer whilst capturing - expected to be an
@@ -347,7 +358,8 @@ static void buffer_cb(struct vchiq_mmal_instance *instance,
&dev->capture.frame_count,
sizeof(dev->capture.frame_count));
}
- if (vchiq_mmal_submit_buffer(instance, port, buf))
+ if (vchiq_mmal_submit_buffer(instance, port,
+ &buf->mmal))
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev,
"Failed to return EOS buffer");
} else {
@@ -367,16 +379,16 @@ static void buffer_cb(struct vchiq_mmal_instance *instance,
return;
}
- if (dev->capture.vc_start_timestamp != -1 && pts) {
+ if (dev->capture.vc_start_timestamp != -1 && mmal_buf->pts) {
ktime_t timestamp;
- s64 runtime_us = pts -
+ s64 runtime_us = mmal_buf->pts -
dev->capture.vc_start_timestamp;
timestamp = ktime_add_us(dev->capture.kernel_start_ts,
runtime_us);
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev,
"Convert start time %llu and %llu with offset %llu to %llu\n",
ktime_to_ns(dev->capture.kernel_start_ts),
- dev->capture.vc_start_timestamp, pts,
+ dev->capture.vc_start_timestamp, mmal_buf->pts,
ktime_to_ns(timestamp));
buf->vb.vb2_buf.timestamp = ktime_to_ns(timestamp);
} else {
@@ -385,13 +397,13 @@ static void buffer_cb(struct vchiq_mmal_instance *instance,
buf->vb.sequence = dev->capture.sequence++;
buf->vb.field = V4L2_FIELD_NONE;
- vb2_set_plane_payload(&buf->vb.vb2_buf, 0, length);
- if (mmal_flags & MMAL_BUFFER_HEADER_FLAG_KEYFRAME)
+ vb2_set_plane_payload(&buf->vb.vb2_buf, 0, mmal_buf->length);
+ if (mmal_buf->mmal_flags & MMAL_BUFFER_HEADER_FLAG_KEYFRAME)
buf->vb.flags |= V4L2_BUF_FLAG_KEYFRAME;
vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE);
- if (mmal_flags & MMAL_BUFFER_HEADER_FLAG_EOS &&
+ if (mmal_buf->mmal_flags & MMAL_BUFFER_HEADER_FLAG_EOS &&
is_capturing(dev)) {
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev,
"Grab another frame as buffer has EOS");
@@ -475,14 +487,16 @@ static void buffer_queue(struct vb2_buffer *vb)
{
struct bm2835_mmal_dev *dev = vb2_get_drv_priv(vb->vb2_queue);
struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
- struct mmal_buffer *buf = container_of(vb2, struct mmal_buffer, vb);
+ struct vb2_mmal_buffer *buf =
+ container_of(vb2, struct vb2_mmal_buffer, vb);
int ret;
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev,
"%s: dev:%p buf:%p, idx %u\n",
__func__, dev, buf, vb2->vb2_buf.index);
- ret = vchiq_mmal_submit_buffer(dev->instance, dev->capture.port, buf);
+ ret = vchiq_mmal_submit_buffer(dev->instance, dev->capture.port,
+ &buf->mmal);
if (ret < 0)
v4l2_err(&dev->v4l2_dev, "%s: error submitting buffer\n",
__func__);
@@ -595,7 +609,7 @@ static void stop_streaming(struct vb2_queue *vq)
dev->capture.frame_count = 0;
/* ensure a format has actually been set */
- if (!dev->capture.port) {
+ if (!port) {
v4l2_err(&dev->v4l2_dev,
"no capture port - stream not started?\n");
return;
@@ -615,11 +629,11 @@ static void stop_streaming(struct vb2_queue *vq)
/* disable the connection from camera to encoder */
ret = vchiq_mmal_port_disable(dev->instance, dev->capture.camera_port);
- if (!ret && dev->capture.camera_port != dev->capture.port) {
+ if (!ret && dev->capture.camera_port != port) {
v4l2_dbg(1, bcm2835_v4l2_debug, &dev->v4l2_dev,
"disabling port\n");
- ret = vchiq_mmal_port_disable(dev->instance, dev->capture.port);
- } else if (dev->capture.camera_port != dev->capture.port) {
+ ret = vchiq_mmal_port_disable(dev->instance, port);
+ } else if (dev->capture.camera_port != port) {
v4l2_err(&dev->v4l2_dev, "port_disable failed, error %d\n",
ret);
}
@@ -1921,7 +1935,7 @@ static int bcm2835_mmal_probe(struct platform_device *pdev)
q->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
q->io_modes = VB2_MMAP | VB2_USERPTR | VB2_READ;
q->drv_priv = dev;
- q->buf_struct_size = sizeof(struct mmal_buffer);
+ q->buf_struct_size = sizeof(struct vb2_mmal_buffer);
q->ops = &bm2835_mmal_video_qops;
q->mem_ops = &vb2_vmalloc_memops;
q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
index ff5398737b4a..a635389208fc 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
@@ -49,6 +49,11 @@ struct mmal_buffer {
unsigned long buffer_size; /* size of allocated buffer */
struct mmal_msg_context *msg_context;
+
+ unsigned long length;
+ u32 mmal_flags;
+ s64 dts;
+ s64 pts;
};
/* */
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 12c2f86b02ad..9af964422e42 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -253,17 +253,25 @@ static void buffer_work_cb(struct work_struct *work)
{
struct mmal_msg_context *msg_context =
container_of(work, struct mmal_msg_context, u.bulk.work);
+ struct mmal_buffer *buffer = msg_context->u.bulk.buffer;
+
+ if (!buffer) {
+ pr_err("%s: ctx: %p, No mmal buffer to pass details\n",
+ __func__, msg_context);
+ return;
+ }
+
+ buffer->length = msg_context->u.bulk.buffer_used;
+ buffer->mmal_flags = msg_context->u.bulk.mmal_flags;
+ buffer->dts = msg_context->u.bulk.dts;
+ buffer->pts = msg_context->u.bulk.pts;
atomic_dec(&msg_context->u.bulk.port->buffers_with_vpu);
msg_context->u.bulk.port->buffer_cb(msg_context->u.bulk.instance,
msg_context->u.bulk.port,
msg_context->u.bulk.status,
- msg_context->u.bulk.buffer,
- msg_context->u.bulk.buffer_used,
- msg_context->u.bulk.mmal_flags,
- msg_context->u.bulk.dts,
- msg_context->u.bulk.pts);
+ msg_context->u.bulk.buffer);
}
/* workqueue scheduled callback to handle receiving buffers
@@ -1321,11 +1329,14 @@ static int port_disable(struct vchiq_mmal_instance *instance,
mmalbuf = list_entry(buf_head, struct mmal_buffer,
list);
list_del(buf_head);
- if (port->buffer_cb)
+ if (port->buffer_cb) {
+ mmalbuf->length = 0;
+ mmalbuf->mmal_flags = 0;
+ mmalbuf->dts = MMAL_TIME_UNKNOWN;
+ mmalbuf->pts = MMAL_TIME_UNKNOWN;
port->buffer_cb(instance,
- port, 0, mmalbuf, 0, 0,
- MMAL_TIME_UNKNOWN,
- MMAL_TIME_UNKNOWN);
+ port, 0, mmalbuf);
+ }
}
spin_unlock_irqrestore(&port->slock, flags);
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index 4e34728d87e5..cca7289761c2 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -44,8 +44,7 @@ struct vchiq_mmal_port;
typedef void (*vchiq_mmal_buffer_cb)(
struct vchiq_mmal_instance *instance,
struct vchiq_mmal_port *port,
- int status, struct mmal_buffer *buffer,
- unsigned long length, u32 mmal_flags, s64 dts, s64 pts);
+ int status, struct mmal_buffer *buffer);
struct vchiq_mmal_port {
u32 enabled:1;
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 12/34] staging: mmal-vchiq: Add support for event callbacks.
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (10 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 11/34] staging: mmal-vchiq: Make a mmal_buf struct for passing parameters Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 13/34] staging: mmal-vchiq: Support sending data to MMAL ports Laurent Pinchart
` (22 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
(Preparation for the codec driver).
The codec uses the event mechanism to report things such as
resolution changes. It is signalled by the cmd field of the buffer
being non-zero.
Add support for passing this information out to the client.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vc04_services/vchiq-mmal/mmal-common.h | 1 +
.../vc04_services/vchiq-mmal/mmal-msg.h | 35 ++++
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 170 ++++++++++++++++--
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 4 +
4 files changed, 196 insertions(+), 14 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
index a635389208fc..b78c180e0c79 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
@@ -50,6 +50,7 @@ struct mmal_buffer {
struct mmal_msg_context *msg_context;
+ u32 cmd; /* MMAL command. 0=data. */
unsigned long length;
u32 mmal_flags;
s64 dts;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
index 43cc59316f90..607ea384f715 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
@@ -346,6 +346,41 @@ struct mmal_msg_port_parameter_get_reply {
/* event messages */
#define MMAL_WORKER_EVENT_SPACE 256
+/* Four CC's for events */
+#define MMAL_FOURCC(a, b, c, d) ((a) | (b << 8) | (c << 16) | (d << 24))
+
+#define MMAL_EVENT_ERROR MMAL_FOURCC('E', 'R', 'R', 'O')
+#define MMAL_EVENT_EOS MMAL_FOURCC('E', 'E', 'O', 'S')
+#define MMAL_EVENT_FORMAT_CHANGED MMAL_FOURCC('E', 'F', 'C', 'H')
+#define MMAL_EVENT_PARAMETER_CHANGED MMAL_FOURCC('E', 'P', 'C', 'H')
+
+/* Structs for each of the event message payloads */
+struct mmal_msg_event_eos {
+ u32 port_type; /**< Type of port that received the end of stream */
+ u32 port_index; /**< Index of port that received the end of stream */
+};
+
+/** Format changed event data. */
+struct mmal_msg_event_format_changed {
+ /* Minimum size of buffers the port requires */
+ u32 buffer_size_min;
+ /* Minimum number of buffers the port requires */
+ u32 buffer_num_min;
+ /* Size of buffers the port recommends for optimal performance.
+ * A value of zero means no special recommendation.
+ */
+ u32 buffer_size_recommended;
+ /* Number of buffers the port recommends for optimal
+ * performance. A value of zero means no special recommendation.
+ */
+ u32 buffer_num_recommended;
+
+ u32 es_ptr;
+ struct mmal_es_format format;
+ union mmal_es_specific_format es;
+ u8 extradata[MMAL_FORMAT_EXTRADATA_MAX_SIZE];
+};
+
struct mmal_msg_event_to_host {
u32 client_component; /* component context */
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 9af964422e42..8358e25851c1 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -145,6 +145,8 @@ struct mmal_msg_context {
/* Presentation and Decode timestamps */
s64 pts;
s64 dts;
+ /* MMAL buffer command flag */
+ u32 cmd;
int status; /* context status */
@@ -232,18 +234,6 @@ release_msg_context(struct mmal_msg_context *msg_context)
kfree(msg_context);
}
-/* deals with receipt of event to host message */
-static void event_to_host_cb(struct vchiq_mmal_instance *instance,
- struct mmal_msg *msg, u32 msg_len)
-{
- pr_debug("unhandled event\n");
- pr_debug("component:%u port type:%d num:%d cmd:0x%x length:%d\n",
- msg->u.event_to_host.client_component,
- msg->u.event_to_host.port_type,
- msg->u.event_to_host.port_num,
- msg->u.event_to_host.cmd, msg->u.event_to_host.length);
-}
-
/* workqueue scheduled callback
*
* we do this because it is important we do not call any other vchiq
@@ -265,13 +255,18 @@ static void buffer_work_cb(struct work_struct *work)
buffer->mmal_flags = msg_context->u.bulk.mmal_flags;
buffer->dts = msg_context->u.bulk.dts;
buffer->pts = msg_context->u.bulk.pts;
+ buffer->cmd = msg_context->u.bulk.cmd;
- atomic_dec(&msg_context->u.bulk.port->buffers_with_vpu);
+ if (!buffer->cmd)
+ atomic_dec(&msg_context->u.bulk.port->buffers_with_vpu);
msg_context->u.bulk.port->buffer_cb(msg_context->u.bulk.instance,
msg_context->u.bulk.port,
msg_context->u.bulk.status,
msg_context->u.bulk.buffer);
+
+ if (buffer->cmd)
+ mutex_unlock(&msg_context->u.bulk.port->event_context_mutex);
}
/* workqueue scheduled callback to handle receiving buffers
@@ -350,6 +345,7 @@ static int bulk_receive(struct vchiq_mmal_instance *instance,
msg_context->u.bulk.buffer_used = rd_len;
msg_context->u.bulk.dts = msg->u.buffer_from_host.buffer_header.dts;
msg_context->u.bulk.pts = msg->u.buffer_from_host.buffer_header.pts;
+ msg_context->u.bulk.cmd = msg->u.buffer_from_host.buffer_header.cmd;
queue_work(msg_context->instance->bulk_wq,
&msg_context->u.bulk.buffer_to_host_work);
@@ -451,6 +447,103 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
return ret;
}
+/* deals with receipt of event to host message */
+static void event_to_host_cb(struct vchiq_mmal_instance *instance,
+ struct mmal_msg *msg, u32 msg_len)
+{
+ /* FIXME: Not going to work on 64 bit */
+ struct vchiq_mmal_component *component =
+ (struct vchiq_mmal_component *)msg->u.event_to_host.client_component;
+ struct vchiq_mmal_port *port = NULL;
+ struct mmal_msg_context *msg_context;
+ u32 port_num = msg->u.event_to_host.port_num;
+
+ if (msg->u.buffer_from_host.drvbuf.magic == MMAL_MAGIC) {
+ pr_err("%s: MMAL_MSG_TYPE_BUFFER_TO_HOST with bad magic\n",
+ __func__);
+ return;
+ }
+
+ switch (msg->u.event_to_host.port_type) {
+ case MMAL_PORT_TYPE_CONTROL:
+ if (port_num) {
+ pr_err("%s: port_num of %u >= number of ports 1",
+ __func__, port_num);
+ return;
+ }
+ port = &component->control;
+ break;
+ case MMAL_PORT_TYPE_INPUT:
+ if (port_num >= component->inputs) {
+ pr_err("%s: port_num of %u >= number of ports %u",
+ __func__, port_num,
+ port_num >= component->inputs);
+ return;
+ }
+ port = &component->input[port_num];
+ break;
+ case MMAL_PORT_TYPE_OUTPUT:
+ if (port_num >= component->outputs) {
+ pr_err("%s: port_num of %u >= number of ports %u",
+ __func__, port_num,
+ port_num >= component->outputs);
+ return;
+ }
+ port = &component->output[port_num];
+ break;
+ case MMAL_PORT_TYPE_CLOCK:
+ if (port_num >= component->clocks) {
+ pr_err("%s: port_num of %u >= number of ports %u",
+ __func__, port_num,
+ port_num >= component->clocks);
+ return;
+ }
+ port = &component->clock[port_num];
+ break;
+ default:
+ break;
+ }
+
+ if (!mutex_trylock(&port->event_context_mutex)) {
+ pr_err("dropping event 0x%x\n", msg->u.event_to_host.cmd);
+ return;
+ }
+ msg_context = port->event_context;
+
+ if (msg->h.status != MMAL_MSG_STATUS_SUCCESS) {
+ /* message reception had an error */
+ //pr_warn
+ pr_err("%s: error %d in reply\n", __func__, msg->h.status);
+
+ msg_context->u.bulk.status = msg->h.status;
+ } else if (msg->u.event_to_host.length > MMAL_WORKER_EVENT_SPACE) {
+ /* data is not in message, queue a bulk receive */
+ pr_err("%s: payload not in message - bulk receive??! NOT SUPPORTED\n",
+ __func__);
+ msg_context->u.bulk.status = -1;
+ } else {
+ memcpy(msg_context->u.bulk.buffer->buffer,
+ msg->u.event_to_host.data,
+ msg->u.event_to_host.length);
+
+ msg_context->u.bulk.buffer_used =
+ msg->u.event_to_host.length;
+
+ msg_context->u.bulk.mmal_flags = 0;
+ msg_context->u.bulk.dts = MMAL_TIME_UNKNOWN;
+ msg_context->u.bulk.pts = MMAL_TIME_UNKNOWN;
+ msg_context->u.bulk.cmd = msg->u.event_to_host.cmd;
+
+ pr_debug("event component:%u port type:%d num:%d cmd:0x%x length:%d\n",
+ msg->u.event_to_host.client_component,
+ msg->u.event_to_host.port_type,
+ msg->u.event_to_host.port_num,
+ msg->u.event_to_host.cmd, msg->u.event_to_host.length);
+ }
+
+ schedule_work(&msg_context->u.bulk.work);
+}
+
/* deals with receipt of buffer to host message */
static void buffer_to_host_cb(struct vchiq_mmal_instance *instance,
struct mmal_msg *msg, u32 msg_len)
@@ -1334,6 +1427,7 @@ static int port_disable(struct vchiq_mmal_instance *instance,
mmalbuf->mmal_flags = 0;
mmalbuf->dts = MMAL_TIME_UNKNOWN;
mmalbuf->pts = MMAL_TIME_UNKNOWN;
+ mmalbuf->cmd = 0;
port->buffer_cb(instance,
port, 0, mmalbuf);
}
@@ -1635,6 +1729,43 @@ int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
}
EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
+static void init_event_context(struct vchiq_mmal_instance *instance,
+ struct vchiq_mmal_port *port)
+{
+ struct mmal_msg_context *ctx = get_msg_context(instance);
+
+ mutex_init(&port->event_context_mutex);
+
+ port->event_context = ctx;
+ ctx->u.bulk.instance = instance;
+ ctx->u.bulk.port = port;
+ ctx->u.bulk.buffer =
+ kzalloc(sizeof(*ctx->u.bulk.buffer), GFP_KERNEL);
+ if (!ctx->u.bulk.buffer)
+ goto release_msg_context;
+ ctx->u.bulk.buffer->buffer = kzalloc(MMAL_WORKER_EVENT_SPACE,
+ GFP_KERNEL);
+ if (!ctx->u.bulk.buffer->buffer)
+ goto release_buffer;
+
+ INIT_WORK(&ctx->u.bulk.work, buffer_work_cb);
+ return;
+
+release_buffer:
+ kfree(ctx->u.bulk.buffer);
+release_msg_context:
+ release_msg_context(ctx);
+}
+
+static void free_event_context(struct vchiq_mmal_port *port)
+{
+ struct mmal_msg_context *ctx = port->event_context;
+
+ kfree(ctx->u.bulk.buffer->buffer);
+ kfree(ctx->u.bulk.buffer);
+ release_msg_context(ctx);
+}
+
/* Initialise a mmal component and its ports
*
*/
@@ -1678,6 +1809,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->control);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->control);
for (idx = 0; idx < component->inputs; idx++) {
component->input[idx].type = MMAL_PORT_TYPE_INPUT;
@@ -1688,6 +1820,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->input[idx]);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->input[idx]);
}
for (idx = 0; idx < component->outputs; idx++) {
@@ -1699,6 +1832,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->output[idx]);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->output[idx]);
}
for (idx = 0; idx < component->clocks; idx++) {
@@ -1710,6 +1844,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
ret = port_info_get(instance, &component->clock[idx]);
if (ret < 0)
goto release_component;
+ init_event_context(instance, &component->clock[idx]);
}
*component_out = component;
@@ -1735,7 +1870,7 @@ EXPORT_SYMBOL_GPL(vchiq_mmal_component_init);
int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_component *component)
{
- int ret;
+ int ret, idx;
if (mutex_lock_interruptible(&instance->vchiq_mutex))
return -EINTR;
@@ -1747,6 +1882,13 @@ int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
component->in_use = 0;
+ for (idx = 0; idx < component->inputs; idx++)
+ free_event_context(&component->input[idx]);
+ for (idx = 0; idx < component->outputs; idx++)
+ free_event_context(&component->output[idx]);
+ for (idx = 0; idx < component->clocks; idx++)
+ free_event_context(&component->clock[idx]);
+
mutex_unlock(&instance->vchiq_mutex);
return ret;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index cca7289761c2..eeef5ee9467a 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -78,6 +78,10 @@ struct vchiq_mmal_port {
vchiq_mmal_buffer_cb buffer_cb;
/* callback context */
void *cb_ctx;
+
+ /* ensure serialised use of the one event context structure */
+ struct mutex event_context_mutex;
+ struct mmal_msg_context *event_context;
};
struct vchiq_mmal_component {
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 13/34] staging: mmal-vchiq: Support sending data to MMAL ports
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (11 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 12/34] staging: mmal-vchiq: Add support for event callbacks Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 14/34] staging: mmal-vchiq: Fixup vchiq-mmal include ordering Laurent Pinchart
` (21 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
Add the ability to send data to ports. This only supports
zero copy mode as the required bulk transfer setup calls
are not done.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 8358e25851c1..63a46818184f 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -422,11 +422,19 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
m.u.buffer_from_host.buffer_header.data =
(u32)(unsigned long)buf->buffer;
m.u.buffer_from_host.buffer_header.alloc_size = buf->buffer_size;
- m.u.buffer_from_host.buffer_header.length = 0; /* nothing used yet */
- m.u.buffer_from_host.buffer_header.offset = 0; /* no offset */
- m.u.buffer_from_host.buffer_header.flags = 0; /* no flags */
- m.u.buffer_from_host.buffer_header.pts = MMAL_TIME_UNKNOWN;
- m.u.buffer_from_host.buffer_header.dts = MMAL_TIME_UNKNOWN;
+ if (port->type == MMAL_PORT_TYPE_OUTPUT) {
+ m.u.buffer_from_host.buffer_header.length = 0;
+ m.u.buffer_from_host.buffer_header.offset = 0;
+ m.u.buffer_from_host.buffer_header.flags = 0;
+ m.u.buffer_from_host.buffer_header.pts = MMAL_TIME_UNKNOWN;
+ m.u.buffer_from_host.buffer_header.dts = MMAL_TIME_UNKNOWN;
+ } else {
+ m.u.buffer_from_host.buffer_header.length = buf->length;
+ m.u.buffer_from_host.buffer_header.offset = 0;
+ m.u.buffer_from_host.buffer_header.flags = buf->mmal_flags;
+ m.u.buffer_from_host.buffer_header.pts = buf->pts;
+ m.u.buffer_from_host.buffer_header.dts = buf->dts;
+ }
/* clear buffer type sepecific data */
memset(&m.u.buffer_from_host.buffer_header_type_specific, 0,
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 14/34] staging: mmal-vchiq: Fixup vchiq-mmal include ordering
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (12 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 13/34] staging: mmal-vchiq: Support sending data to MMAL ports Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Laurent Pinchart
` (20 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
There were dependencies on including the headers in the correct
order. Fix up the headers so that they include the other
headers that they depend on themselves.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h | 1 +
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
index 607ea384f715..bd83159403e5 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-msg.h
@@ -38,6 +38,7 @@
#include "mmal-msg-common.h"
#include "mmal-msg-format.h"
#include "mmal-msg-port.h"
+#include "mmal-vchiq.h"
enum mmal_msg_type {
MMAL_MSG_TYPE_QUIT = 1,
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index eeef5ee9467a..7a490f29737a 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -16,6 +16,7 @@
#ifndef MMAL_VCHIQ_H
#define MMAL_VCHIQ_H
+#include "mmal-common.h"
#include "mmal-msg-format.h"
#define MAX_PORT_COUNT 4
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (13 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 14/34] staging: mmal-vchiq: Fixup vchiq-mmal include ordering Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 16:30 ` Nicolas Saenz Julienne
` (2 more replies)
2020-05-04 9:25 ` [PATCH v2 16/34] staging: mmal-vchiq: Fix client_component for 64 bit kernel Laurent Pinchart
` (19 subsequent siblings)
34 siblings, 3 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
With the vc-sm-cma driver we can support zero copy of buffers between
the kernel and VPU. Add this support to vchiq-mmal.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
staging: vc-sm-cma: Use a void* pointer as the handle within the kernel
The driver was using an unsigned int as the handle to the outside world,
and doing a nasty cast to the struct dmabuf when handed it back.
This breaks badly with a 64 bit kernel where the pointer doesn't fit
in an unsigned int.
Switch to using a void* within the kernel. Reality is that it is
a struct dma_buf*, but advertising it as such to other drivers seems
to encourage the use of it as such, and I'm not sure on the implications
of that.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vc04_services/vchiq-mmal/mmal-common.h | 4 ++
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 66 ++++++++++++++++++-
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
3 files changed, 69 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
index b78c180e0c79..3893a280fcfa 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-common.h
@@ -50,6 +50,10 @@ struct mmal_buffer {
struct mmal_msg_context *msg_context;
+ struct dma_buf *dma_buf;/* Exported dmabuf fd from videobuf2 */
+ void *vcsm_handle; /* VCSM handle having imported the dmabuf */
+ u32 vc_handle; /* VC handle to that dmabuf */
+
u32 cmd; /* MMAL command. 0=data. */
unsigned long length;
u32 mmal_flags;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 63a46818184f..c65c262cffbb 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -26,9 +26,12 @@
#include <media/videobuf2-vmalloc.h>
#include "mmal-common.h"
+#include "mmal-parameters.h"
#include "mmal-vchiq.h"
#include "mmal-msg.h"
+#include "vc-sm-cma/vc_sm_knl.h"
+
#define USE_VCHIQ_ARM
#include "interface/vchi/vchi.h"
@@ -419,8 +422,13 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
/* buffer header */
m.u.buffer_from_host.buffer_header.cmd = 0;
- m.u.buffer_from_host.buffer_header.data =
- (u32)(unsigned long)buf->buffer;
+ if (port->zero_copy) {
+ m.u.buffer_from_host.buffer_header.data = buf->vc_handle;
+ } else {
+ m.u.buffer_from_host.buffer_header.data =
+ (u32)(unsigned long)buf->buffer;
+ }
+
m.u.buffer_from_host.buffer_header.alloc_size = buf->buffer_size;
if (port->type == MMAL_PORT_TYPE_OUTPUT) {
m.u.buffer_from_host.buffer_header.length = 0;
@@ -585,6 +593,22 @@ static void buffer_to_host_cb(struct vchiq_mmal_instance *instance,
msg_context->u.bulk.status = msg->h.status;
+ } else if (msg->u.buffer_from_host.is_zero_copy) {
+ /*
+ * Zero copy buffer, so nothing to do.
+ * Copy buffer info and make callback.
+ */
+ msg_context->u.bulk.buffer_used =
+ msg->u.buffer_from_host.buffer_header.length;
+ msg_context->u.bulk.mmal_flags =
+ msg->u.buffer_from_host.buffer_header.flags;
+ msg_context->u.bulk.dts =
+ msg->u.buffer_from_host.buffer_header.dts;
+ msg_context->u.bulk.pts =
+ msg->u.buffer_from_host.buffer_header.pts;
+ msg_context->u.bulk.cmd =
+ msg->u.buffer_from_host.buffer_header.cmd;
+
} else if (msg->u.buffer_from_host.buffer_header.length == 0) {
/* empty buffer */
if (msg->u.buffer_from_host.buffer_header.flags &
@@ -1532,6 +1556,9 @@ int vchiq_mmal_port_parameter_set(struct vchiq_mmal_instance *instance,
mutex_unlock(&instance->vchiq_mutex);
+ if (parameter == MMAL_PARAMETER_ZERO_COPY && !ret)
+ port->zero_copy = !!(*(bool *)value);
+
return ret;
}
EXPORT_SYMBOL_GPL(vchiq_mmal_port_parameter_set);
@@ -1700,6 +1727,31 @@ int vchiq_mmal_submit_buffer(struct vchiq_mmal_instance *instance,
unsigned long flags = 0;
int ret;
+ /*
+ * We really want to do this in mmal_vchi_buffer_init but can't as
+ * videobuf2 won't let us have the dmabuf there.
+ */
+ if (port->zero_copy && buffer->dma_buf && !buffer->vcsm_handle) {
+ pr_debug("%s: import dmabuf %p\n", __func__, buffer->dma_buf);
+ ret = vc_sm_cma_import_dmabuf(buffer->dma_buf,
+ &buffer->vcsm_handle);
+ if (ret) {
+ pr_err("%s: vc_sm_import_dmabuf_fd failed, ret %d\n",
+ __func__, ret);
+ return ret;
+ }
+
+ buffer->vc_handle = vc_sm_cma_int_handle(buffer->vcsm_handle);
+ if (!buffer->vc_handle) {
+ pr_err("%s: vc_sm_int_handle failed %d\n",
+ __func__, ret);
+ vc_sm_cma_free(buffer->vcsm_handle);
+ return ret;
+ }
+ pr_debug("%s: import dmabuf %p - got vc handle %08X\n",
+ __func__, buffer->dma_buf, buffer->vc_handle);
+ }
+
ret = buffer_from_host(instance, port, buffer);
if (ret == -EINVAL) {
/* Port is disabled. Queue for when it is enabled. */
@@ -1733,6 +1785,16 @@ int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
release_msg_context(msg_context);
buf->msg_context = NULL;
+ if (buf->vcsm_handle) {
+ int ret;
+
+ pr_debug("%s: vc_sm_cma_free on handle %p\n", __func__,
+ buf->vcsm_handle);
+ ret = vc_sm_cma_free(buf->vcsm_handle);
+ if (ret)
+ pr_err("%s: vcsm_free failed, ret %d\n", __func__, ret);
+ buf->vcsm_handle = 0;
+ }
return 0;
}
EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index 7a490f29737a..ae6c69ba16ee 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -49,6 +49,7 @@ typedef void (*vchiq_mmal_buffer_cb)(
struct vchiq_mmal_port {
u32 enabled:1;
+ u32 zero_copy:1;
u32 handle;
u32 type; /* port type, cached to use on port info set */
u32 index; /* port index, cached to use on port info set */
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
2020-05-04 9:25 ` [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Laurent Pinchart
@ 2020-05-04 16:30 ` Nicolas Saenz Julienne
2020-05-05 16:07 ` kbuild test robot
2020-05-11 19:15 ` Nicolas Saenz Julienne
2 siblings, 0 replies; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-04 16:30 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 1074 bytes --]
On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> With the vc-sm-cma driver we can support zero copy of buffers between
> the kernel and VPU. Add this support to vchiq-mmal.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> staging: vc-sm-cma: Use a void* pointer as the handle within the kernel
>
> The driver was using an unsigned int as the handle to the outside world,
> and doing a nasty cast to the struct dmabuf when handed it back.
> This breaks badly with a 64 bit kernel where the pointer doesn't fit
> in an unsigned int.
>
> Switch to using a void* within the kernel. Reality is that it is
> a struct dma_buf*, but advertising it as such to other drivers seems
> to encourage the use of it as such, and I'm not sure on the implications
> of that.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
I think this patch's description needs to be updated.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
2020-05-04 9:25 ` [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Laurent Pinchart
@ 2020-05-05 16:07 ` kbuild test robot
2020-05-05 16:07 ` kbuild test robot
2020-05-11 19:15 ` Nicolas Saenz Julienne
2 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 16:07 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: kbuild-all, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Laurent,
I love your patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
reproduce:
# apt-get install sparse
# sparse version: v0.6.1-191-gc51a0382-dirty
make ARCH=x86_64 allmodconfig
make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
sparse warnings: (new ones prefixed by >>)
>> drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c:1796:36: sparse: sparse: Using plain integer as NULL pointer
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c:472:48: sparse: sparse: non size-preserving integer to pointer cast
vim +1796 drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
1779
1780 int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
1781 {
1782 struct mmal_msg_context *msg_context = buf->msg_context;
1783
1784 if (msg_context)
1785 release_msg_context(msg_context);
1786 buf->msg_context = NULL;
1787
1788 if (buf->vcsm_handle) {
1789 int ret;
1790
1791 pr_debug("%s: vc_sm_cma_free on handle %p\n", __func__,
1792 buf->vcsm_handle);
1793 ret = vc_sm_cma_free(buf->vcsm_handle);
1794 if (ret)
1795 pr_err("%s: vcsm_free failed, ret %d\n", __func__, ret);
> 1796 buf->vcsm_handle = 0;
1797 }
1798 return 0;
1799 }
1800 EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
1801
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
@ 2020-05-05 16:07 ` kbuild test robot
0 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 16:07 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2208 bytes --]
Hi Laurent,
I love your patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
reproduce:
# apt-get install sparse
# sparse version: v0.6.1-191-gc51a0382-dirty
make ARCH=x86_64 allmodconfig
make C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__'
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
sparse warnings: (new ones prefixed by >>)
>> drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c:1796:36: sparse: sparse: Using plain integer as NULL pointer
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c:472:48: sparse: sparse: non size-preserving integer to pointer cast
vim +1796 drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
1779
1780 int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
1781 {
1782 struct mmal_msg_context *msg_context = buf->msg_context;
1783
1784 if (msg_context)
1785 release_msg_context(msg_context);
1786 buf->msg_context = NULL;
1787
1788 if (buf->vcsm_handle) {
1789 int ret;
1790
1791 pr_debug("%s: vc_sm_cma_free on handle %p\n", __func__,
1792 buf->vcsm_handle);
1793 ret = vc_sm_cma_free(buf->vcsm_handle);
1794 if (ret)
1795 pr_err("%s: vcsm_free failed, ret %d\n", __func__, ret);
> 1796 buf->vcsm_handle = 0;
1797 }
1798 return 0;
1799 }
1800 EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
1801
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy
2020-05-04 9:25 ` [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Laurent Pinchart
2020-05-04 16:30 ` Nicolas Saenz Julienne
2020-05-05 16:07 ` kbuild test robot
@ 2020-05-11 19:15 ` Nicolas Saenz Julienne
2 siblings, 0 replies; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-11 19:15 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]
Hi Dave,
some questions.
On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> With the vc-sm-cma driver we can support zero copy of buffers between
> the kernel and VPU. Add this support to vchiq-mmal.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> staging: vc-sm-cma: Use a void* pointer as the handle within the kernel
>
> The driver was using an unsigned int as the handle to the outside world,
> and doing a nasty cast to the struct dmabuf when handed it back.
> This breaks badly with a 64 bit kernel where the pointer doesn't fit
> in an unsigned int.
>
> Switch to using a void* within the kernel. Reality is that it is
> a struct dma_buf*, but advertising it as such to other drivers seems
> to encourage the use of it as such, and I'm not sure on the implications
> of that.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
[...]
> @@ -419,8 +422,13 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
>
> /* buffer header */
> m.u.buffer_from_host.buffer_header.cmd = 0;
> - m.u.buffer_from_host.buffer_header.data =
> - (u32)(unsigned long)buf->buffer;
> + if (port->zero_copy) {
> + m.u.buffer_from_host.buffer_header.data = buf->vc_handle;
> + } else {
> + m.u.buffer_from_host.buffer_header.data =
> + (u32)(unsigned long)buf->buffer;
> + }
Just for my education, where exactly does the copying happen, IIUC this headers
are copied into a vchiq slot and then it's up to vc4 to do whatever it pleases
with the buffer. Also, what happens while importing a dmabuf on vc4's side and
why is the buffer import needed before queueing this?
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 16/34] staging: mmal-vchiq: Fix client_component for 64 bit kernel
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (14 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 15/34] staging: mmal-vchiq: Use vc-sm-cma to support zero copy Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-18 10:19 ` Hans Verkuil
2020-05-04 9:25 ` [PATCH v2 17/34] staging: mmal_vchiq: Add in the Bayer encoding formats Laurent Pinchart
` (18 subsequent siblings)
34 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
The MMAL client_component field is used with the event
mechanism to allow the client to identify the component for
which the event is generated.
The field is only 32bits in size, therefore we can't use a
pointer to the component in a 64 bit kernel.
Component handles are already held in an array per VCHI
instance, so use the array index as the client_component handle
to avoid having to create a new IDR for this purpose.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../staging/vc04_services/vchiq-mmal/mmal-vchiq.c | 12 +++++++++---
.../staging/vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index c65c262cffbb..5872690b404d 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -467,9 +467,9 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
static void event_to_host_cb(struct vchiq_mmal_instance *instance,
struct mmal_msg *msg, u32 msg_len)
{
- /* FIXME: Not going to work on 64 bit */
+ int comp_idx = msg->u.event_to_host.client_component;
struct vchiq_mmal_component *component =
- (struct vchiq_mmal_component *)msg->u.event_to_host.client_component;
+ &instance->component[comp_idx];
struct vchiq_mmal_port *port = NULL;
struct mmal_msg_context *msg_context;
u32 port_num = msg->u.event_to_host.port_num;
@@ -1068,7 +1068,7 @@ static int create_component(struct vchiq_mmal_instance *instance,
/* build component create message */
m.h.type = MMAL_MSG_TYPE_COMPONENT_CREATE;
- m.u.component_create.client_component = (u32)(unsigned long)component;
+ m.u.component_create.client_component = component->client_component;
strncpy(m.u.component_create.name, name,
sizeof(m.u.component_create.name));
@@ -1863,6 +1863,12 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
goto unlock;
}
+ /* We need a handle to reference back to our component structure.
+ * Use the array index in instance->component rather than rolling
+ * another IDR.
+ */
+ component->client_component = idx;
+
ret = create_component(instance, component, name);
if (ret < 0) {
pr_err("%s: failed to create component %d (Not enough GPU mem?)\n",
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index ae6c69ba16ee..247521fbcc1d 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -97,6 +97,7 @@ struct vchiq_mmal_component {
struct vchiq_mmal_port input[MAX_PORT_COUNT]; /* input ports */
struct vchiq_mmal_port output[MAX_PORT_COUNT]; /* output ports */
struct vchiq_mmal_port clock[MAX_PORT_COUNT]; /* clock ports */
+ u32 client_component; /* Used to ref back to client struct */
};
int vchiq_mmal_init(struct vchiq_mmal_instance **out_instance);
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 16/34] staging: mmal-vchiq: Fix client_component for 64 bit kernel
2020-05-04 9:25 ` [PATCH v2 16/34] staging: mmal-vchiq: Fix client_component for 64 bit kernel Laurent Pinchart
@ 2020-05-18 10:19 ` Hans Verkuil
0 siblings, 0 replies; 111+ messages in thread
From: Hans Verkuil @ 2020-05-18 10:19 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
On 04/05/2020 11:25, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> The MMAL client_component field is used with the event
> mechanism to allow the client to identify the component for
> which the event is generated.
> The field is only 32bits in size, therefore we can't use a
> pointer to the component in a 64 bit kernel.
>
> Component handles are already held in an array per VCHI
> instance, so use the array index as the client_component handle
> to avoid having to create a new IDR for this purpose.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
> .../staging/vc04_services/vchiq-mmal/mmal-vchiq.c | 12 +++++++++---
> .../staging/vc04_services/vchiq-mmal/mmal-vchiq.h | 1 +
> 2 files changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
> index c65c262cffbb..5872690b404d 100644
> --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
> +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
> @@ -467,9 +467,9 @@ buffer_from_host(struct vchiq_mmal_instance *instance,
> static void event_to_host_cb(struct vchiq_mmal_instance *instance,
> struct mmal_msg *msg, u32 msg_len)
> {
> - /* FIXME: Not going to work on 64 bit */
> + int comp_idx = msg->u.event_to_host.client_component;
> struct vchiq_mmal_component *component =
> - (struct vchiq_mmal_component *)msg->u.event_to_host.client_component;
> + &instance->component[comp_idx];
> struct vchiq_mmal_port *port = NULL;
> struct mmal_msg_context *msg_context;
> u32 port_num = msg->u.event_to_host.port_num;
> @@ -1068,7 +1068,7 @@ static int create_component(struct vchiq_mmal_instance *instance,
>
> /* build component create message */
> m.h.type = MMAL_MSG_TYPE_COMPONENT_CREATE;
> - m.u.component_create.client_component = (u32)(unsigned long)component;
> + m.u.component_create.client_component = component->client_component;
> strncpy(m.u.component_create.name, name,
> sizeof(m.u.component_create.name));
I recommend that another patch is made that replaces any strcpy/strlcpy/strncpy
calls to strscpy (since that's the standard for media drivers).
Regards,
Hans
>
> @@ -1863,6 +1863,12 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
> goto unlock;
> }
>
> + /* We need a handle to reference back to our component structure.
> + * Use the array index in instance->component rather than rolling
> + * another IDR.
> + */
> + component->client_component = idx;
> +
> ret = create_component(instance, component, name);
> if (ret < 0) {
> pr_err("%s: failed to create component %d (Not enough GPU mem?)\n",
> diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
> index ae6c69ba16ee..247521fbcc1d 100644
> --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
> +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
> @@ -97,6 +97,7 @@ struct vchiq_mmal_component {
> struct vchiq_mmal_port input[MAX_PORT_COUNT]; /* input ports */
> struct vchiq_mmal_port output[MAX_PORT_COUNT]; /* output ports */
> struct vchiq_mmal_port clock[MAX_PORT_COUNT]; /* clock ports */
> + u32 client_component; /* Used to ref back to client struct */
> };
>
> int vchiq_mmal_init(struct vchiq_mmal_instance **out_instance);
>
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 17/34] staging: mmal_vchiq: Add in the Bayer encoding formats
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (15 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 16/34] staging: mmal-vchiq: Fix client_component for 64 bit kernel Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 18/34] staging: mmal-vchiq: Always return the param size from param_get Laurent Pinchart
` (17 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
The list of formats was copied before Bayer support was added.
The ISP supports Bayer and is being supported by the bcm2835_codec
driver, so add in the encodings for them.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vc04_services/vchiq-mmal/mmal-encodings.h | 27 +++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
index 2be9941a1f30..44ba91aa6d47 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
@@ -69,6 +69,33 @@
*/
#define MMAL_ENCODING_OPAQUE MMAL_FOURCC('O', 'P', 'Q', 'V')
+/* Bayer formats
+ * FourCC values copied from V4L2 where defined.
+ */
+/* 8 bit per pixel Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR8 MMAL_FOURCC('B', 'A', '8', '1')
+#define MMAL_ENCODING_BAYER_SGBRG8 MMAL_FOURCC('G', 'B', 'R', 'G')
+#define MMAL_ENCODING_BAYER_SGRBG8 MMAL_FOURCC('G', 'R', 'B', 'G')
+#define MMAL_ENCODING_BAYER_SRGGB8 MMAL_FOURCC('R', 'G', 'G', 'B')
+
+/* 10 bit per pixel packed Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR10P MMAL_FOURCC('p', 'B', 'A', 'A')
+#define MMAL_ENCODING_BAYER_SGRBG10P MMAL_FOURCC('p', 'g', 'A', 'A')
+#define MMAL_ENCODING_BAYER_SGBRG10P MMAL_FOURCC('p', 'G', 'A', 'A')
+#define MMAL_ENCODING_BAYER_SRGGB10P MMAL_FOURCC('p', 'R', 'A', 'A')
+
+/* 12 bit per pixel packed Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR12P MMAL_FOURCC('p', 'B', '1', '2')
+#define MMAL_ENCODING_BAYER_SGRBG12P MMAL_FOURCC('p', 'g', '1', '2')
+#define MMAL_ENCODING_BAYER_SGBRG12P MMAL_FOURCC('p', 'G', '1', '2')
+#define MMAL_ENCODING_BAYER_SRGGB12P MMAL_FOURCC('p', 'R', '1', '2')
+
+/* 16 bit per pixel Bayer formats. */
+#define MMAL_ENCODING_BAYER_SBGGR16 MMAL_FOURCC('B', 'G', '1', '6')
+#define MMAL_ENCODING_BAYER_SGBRG16 MMAL_FOURCC('G', 'B', '1', '6')
+#define MMAL_ENCODING_BAYER_SGRBG16 MMAL_FOURCC('G', 'R', '1', '6')
+#define MMAL_ENCODING_BAYER_SRGGB16 MMAL_FOURCC('R', 'G', '1', '6')
+
/** An EGL image handle
*/
#define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 18/34] staging: mmal-vchiq: Always return the param size from param_get
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (16 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 17/34] staging: mmal_vchiq: Add in the Bayer encoding formats Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 19/34] staging: mmal-vchiq: If the VPU returns an error, don't negate it Laurent Pinchart
` (16 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
mmal-vchiq is a reimplementation of the userland library for MMAL.
When getting a parameter, the client provides the storage and
the size of the storage. The VPU then returns the size of the
parameter that it wished to return, and as much as possible of
that parameter is returned to the client.
The implementation previously only returned the size provided
by the VPU should it exceed the buffer size. So for parameters
such as the supported encodings list the client had no idea
how much of the provided storage had been populated.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 5872690b404d..5e7d4263e39b 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -1407,11 +1407,12 @@ static int port_parameter_get(struct vchiq_mmal_instance *instance,
*/
memcpy(value, &rmsg->u.port_parameter_get_reply.value,
*value_size);
- *value_size = rmsg->u.port_parameter_get_reply.size;
} else {
memcpy(value, &rmsg->u.port_parameter_get_reply.value,
rmsg->u.port_parameter_get_reply.size);
}
+ /* Always report the size of the returned parameter to the caller */
+ *value_size = rmsg->u.port_parameter_get_reply.size;
pr_debug("%s:result:%d component:0x%x port:%d parameter:%d\n", __func__,
ret, port->component->handle, port->handle, parameter_id);
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 19/34] staging: mmal-vchiq: If the VPU returns an error, don't negate it
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (17 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 18/34] staging: mmal-vchiq: Always return the param size from param_get Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 20/34] staging: mmal-vchiq: Fix handling of VB2_MEMORY_DMABUF buffers Laurent Pinchart
` (15 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
There is an enum for the errors that the VPU can return.
port_parameter_get was negating that value, but also using -EINVAL
from the Linux error codes.
Pass the VPU error code as positive values. Should the function
need to pass a Linux failure, then return that as negative.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 5e7d4263e39b..4e559f88d828 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -1395,7 +1395,8 @@ static int port_parameter_get(struct vchiq_mmal_instance *instance,
goto release_msg;
}
- ret = -rmsg->u.port_parameter_get_reply.status;
+ ret = rmsg->u.port_parameter_get_reply.status;
+
/* port_parameter_get_reply.size includes the header,
* whilst *value_size doesn't.
*/
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 20/34] staging: mmal-vchiq: Fix handling of VB2_MEMORY_DMABUF buffers
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (18 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 19/34] staging: mmal-vchiq: If the VPU returns an error, don't negate it Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 21/34] staging: mmal-vchiq: Update mmal_parameters.h with recently defined params Laurent Pinchart
` (14 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
If the queue is configured as VB2_MEMORY_DMABUF then vb2_core_expbuf
fails as it ensures the queue is defined as VB2_MEMORY_MMAP.
Correct the handling so that we unmap the buffer from vcsm and the
VPU on cleanup, and then correctly get the dma buf of the new buffer.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 21 +++++++++++++------
.../vc04_services/vchiq-mmal/mmal-vchiq.h | 2 ++
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 4e559f88d828..9fc1a29f9f1a 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -1779,13 +1779,9 @@ int mmal_vchi_buffer_init(struct vchiq_mmal_instance *instance,
}
EXPORT_SYMBOL_GPL(mmal_vchi_buffer_init);
-int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
+int mmal_vchi_buffer_unmap(struct mmal_buffer *buf)
{
- struct mmal_msg_context *msg_context = buf->msg_context;
-
- if (msg_context)
- release_msg_context(msg_context);
- buf->msg_context = NULL;
+ int ret = 0;
if (buf->vcsm_handle) {
int ret;
@@ -1797,6 +1793,19 @@ int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
pr_err("%s: vcsm_free failed, ret %d\n", __func__, ret);
buf->vcsm_handle = 0;
}
+ return ret;
+}
+EXPORT_SYMBOL_GPL(mmal_vchi_buffer_unmap);
+
+int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf)
+{
+ struct mmal_msg_context *msg_context = buf->msg_context;
+
+ if (msg_context)
+ release_msg_context(msg_context);
+ buf->msg_context = NULL;
+
+ mmal_vchi_buffer_unmap(buf);
return 0;
}
EXPORT_SYMBOL_GPL(mmal_vchi_buffer_cleanup);
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
index 247521fbcc1d..0a75c96f6d58 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.h
@@ -167,6 +167,8 @@ int vchiq_mmal_submit_buffer(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_port *port,
struct mmal_buffer *buf);
+int mmal_vchi_buffer_unmap(struct mmal_buffer *buf);
+
int mmal_vchi_buffer_init(struct vchiq_mmal_instance *instance,
struct mmal_buffer *buf);
int mmal_vchi_buffer_cleanup(struct mmal_buffer *buf);
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 21/34] staging: mmal-vchiq: Update mmal_parameters.h with recently defined params
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (19 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 20/34] staging: mmal-vchiq: Fix handling of VB2_MEMORY_DMABUF buffers Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:25 ` [PATCH v2 22/34] staging: mmal-vchiq: Free the event context for control ports Laurent Pinchart
` (13 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
mmal_parameters.h hasn't been updated to reflect additions made
over the last few years. Update it to reflect the currently
supported parameters.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vchiq-mmal/mmal-parameters.h | 32 ++++++++++++++++++-
1 file changed, 31 insertions(+), 1 deletion(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
index 80a99128f5f3..926392d754c7 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
@@ -580,7 +580,37 @@ enum mmal_parameter_video_type {
MMAL_PARAMETER_VIDEO_ENCODE_H264_LOW_DELAY_HRD_FLAG,
/**< @ref MMAL_PARAMETER_BOOLEAN_T */
- MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER
+ MMAL_PARAMETER_VIDEO_ENCODE_INLINE_HEADER,
+
+ /**< Take a @ref MMAL_PARAMETER_BOOLEAN_T. */
+ MMAL_PARAMETER_VIDEO_ENCODE_SEI_ENABLE,
+
+ /**< Take a @ref MMAL_PARAMETER_BOOLEAN_T. */
+ MMAL_PARAMETER_VIDEO_ENCODE_INLINE_VECTORS,
+
+ /**< Take a @ref MMAL_PARAMETER_VIDEO_RENDER_STATS_T. */
+ MMAL_PARAMETER_VIDEO_RENDER_STATS,
+
+ /**< Take a @ref MMAL_PARAMETER_VIDEO_INTERLACE_TYPE_T. */
+ MMAL_PARAMETER_VIDEO_INTERLACE_TYPE,
+
+ /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
+ MMAL_PARAMETER_VIDEO_INTERPOLATE_TIMESTAMPS,
+
+ /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
+ MMAL_PARAMETER_VIDEO_ENCODE_SPS_TIMING,
+
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_VIDEO_MAX_NUM_CALLBACKS,
+
+ /**< Takes a @ref MMAL_PARAMETER_SOURCE_PATTERN_T */
+ MMAL_PARAMETER_VIDEO_SOURCE_PATTERN,
+
+ /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
+ MMAL_PARAMETER_VIDEO_ENCODE_SEPARATE_NAL_BUFS,
+
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_VIDEO_DROPPABLE_PFRAME_LENGTH,
};
/** Valid mirror modes */
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 22/34] staging: mmal-vchiq: Free the event context for control ports
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (20 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 21/34] staging: mmal-vchiq: Update mmal_parameters.h with recently defined params Laurent Pinchart
@ 2020-05-04 9:25 ` Laurent Pinchart
2020-05-04 9:26 ` [PATCH v2 23/34] staging: mmal-vchiq: Fix memory leak in error path Laurent Pinchart
` (12 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:25 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
vchiq_mmal_component_init calls init_event_context for the
control port, but vchiq_mmal_component_finalise didn't free
it, causing a memory leak..
Add the free call.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 9fc1a29f9f1a..95437d9c519c 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -1976,6 +1976,8 @@ int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
for (idx = 0; idx < component->clocks; idx++)
free_event_context(&component->clock[idx]);
+ free_event_context(&component->control);
+
mutex_unlock(&instance->vchiq_mutex);
return ret;
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 23/34] staging: mmal-vchiq: Fix memory leak in error path
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (21 preceding siblings ...)
2020-05-04 9:25 ` [PATCH v2 22/34] staging: mmal-vchiq: Free the event context for control ports Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 9:26 ` [PATCH v2 24/34] staging: mmal-vchiq: Fix formatting errors in mmal_parameters.h Laurent Pinchart
` (11 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
On error, vchiq_mmal_component_init could leave the
event context allocated for ports.
Clean them up in the error path.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../vc04_services/vchiq-mmal/mmal-vchiq.c | 29 +++++++++++++------
1 file changed, 20 insertions(+), 9 deletions(-)
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
index 95437d9c519c..4c89bc0bec15 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c
@@ -1842,9 +1842,26 @@ static void free_event_context(struct vchiq_mmal_port *port)
{
struct mmal_msg_context *ctx = port->event_context;
+ if (!ctx)
+ return;
+
kfree(ctx->u.bulk.buffer->buffer);
kfree(ctx->u.bulk.buffer);
release_msg_context(ctx);
+ port->event_context = NULL;
+}
+
+static void release_all_event_contexts(struct vchiq_mmal_component *component)
+{
+ int idx;
+
+ for (idx = 0; idx < component->inputs; idx++)
+ free_event_context(&component->input[idx]);
+ for (idx = 0; idx < component->outputs; idx++)
+ free_event_context(&component->output[idx]);
+ for (idx = 0; idx < component->clocks; idx++)
+ free_event_context(&component->clock[idx]);
+ free_event_context(&component->control);
}
/* Initialise a mmal component and its ports
@@ -1942,6 +1959,7 @@ int vchiq_mmal_component_init(struct vchiq_mmal_instance *instance,
release_component:
destroy_component(instance, component);
+ release_all_event_contexts(component);
unlock:
if (component)
component->in_use = 0;
@@ -1957,7 +1975,7 @@ EXPORT_SYMBOL_GPL(vchiq_mmal_component_init);
int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
struct vchiq_mmal_component *component)
{
- int ret, idx;
+ int ret;
if (mutex_lock_interruptible(&instance->vchiq_mutex))
return -EINTR;
@@ -1969,14 +1987,7 @@ int vchiq_mmal_component_finalise(struct vchiq_mmal_instance *instance,
component->in_use = 0;
- for (idx = 0; idx < component->inputs; idx++)
- free_event_context(&component->input[idx]);
- for (idx = 0; idx < component->outputs; idx++)
- free_event_context(&component->output[idx]);
- for (idx = 0; idx < component->clocks; idx++)
- free_event_context(&component->clock[idx]);
-
- free_event_context(&component->control);
+ release_all_event_contexts(component);
mutex_unlock(&instance->vchiq_mutex);
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 24/34] staging: mmal-vchiq: Fix formatting errors in mmal_parameters.h
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (22 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 23/34] staging: mmal-vchiq: Fix memory leak in error path Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 9:26 ` [PATCH v2 25/34] staging: vchiq_arm: Register vcsm-cma as a platform driver Laurent Pinchart
` (10 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Naushir Patuck <naush@raspberrypi.com>
No functional changes in this commit.
- Remove erroneous whitespace.
- Remove _t postfix label on structs and enums.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../bcm2835-camera/bcm2835-camera.c | 2 +-
.../vchiq-mmal/mmal-parameters.h | 46 +++++++++----------
2 files changed, 24 insertions(+), 24 deletions(-)
diff --git a/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c b/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
index 68a2260d88f5..613b04683dca 100644
--- a/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
+++ b/drivers/staging/vc04_services/bcm2835-camera/bcm2835-camera.c
@@ -1497,7 +1497,7 @@ static int get_num_cameras(struct vchiq_mmal_instance *instance,
{
int ret;
struct vchiq_mmal_component *cam_info_component;
- struct mmal_parameter_camera_info_t cam_info = {0};
+ struct mmal_parameter_camera_info cam_info = {0};
u32 param_size = sizeof(cam_info);
int i;
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
index 926392d754c7..1793103b18fd 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
@@ -23,21 +23,21 @@
#define MMAL_PARAMETERS_H
/** Common parameter ID group, used with many types of component. */
-#define MMAL_PARAMETER_GROUP_COMMON (0 << 16)
+#define MMAL_PARAMETER_GROUP_COMMON (0 << 16)
/** Camera-specific parameter ID group. */
-#define MMAL_PARAMETER_GROUP_CAMERA (1 << 16)
+#define MMAL_PARAMETER_GROUP_CAMERA (1 << 16)
/** Video-specific parameter ID group. */
-#define MMAL_PARAMETER_GROUP_VIDEO (2 << 16)
+#define MMAL_PARAMETER_GROUP_VIDEO (2 << 16)
/** Audio-specific parameter ID group. */
-#define MMAL_PARAMETER_GROUP_AUDIO (3 << 16)
+#define MMAL_PARAMETER_GROUP_AUDIO (3 << 16)
/** Clock-specific parameter ID group. */
-#define MMAL_PARAMETER_GROUP_CLOCK (4 << 16)
+#define MMAL_PARAMETER_GROUP_CLOCK (4 << 16)
/** Miracast-specific parameter ID group. */
-#define MMAL_PARAMETER_GROUP_MIRACAST (5 << 16)
+#define MMAL_PARAMETER_GROUP_MIRACAST (5 << 16)
/* Common parameters */
enum mmal_parameter_common_type {
- /**< Never a valid parameter ID */
+ /**< Never a valid parameter ID */
MMAL_PARAMETER_UNUSED = MMAL_PARAMETER_GROUP_COMMON,
/**< MMAL_PARAMETER_ENCODING_T */
@@ -341,7 +341,7 @@ enum mmal_parameter_imagefx {
MMAL_PARAM_IMAGEFX_CARTOON,
};
-enum MMAL_PARAM_FLICKERAVOID_T {
+enum MMAL_PARAM_FLICKERAVOID {
MMAL_PARAM_FLICKERAVOID_OFF,
MMAL_PARAM_FLICKERAVOID_AUTO,
MMAL_PARAM_FLICKERAVOID_50HZ,
@@ -753,15 +753,15 @@ struct mmal_parameter_imagefx_parameters {
#define MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES 2
#define MMAL_PARAMETER_CAMERA_INFO_MAX_STR_LEN 16
-struct mmal_parameter_camera_info_camera_t {
- u32 port_id;
- u32 max_width;
- u32 max_height;
- u32 lens_present;
- u8 camera_name[MMAL_PARAMETER_CAMERA_INFO_MAX_STR_LEN];
+struct mmal_parameter_camera_info_camera {
+ u32 port_id;
+ u32 max_width;
+ u32 max_height;
+ u32 lens_present;
+ u8 camera_name[MMAL_PARAMETER_CAMERA_INFO_MAX_STR_LEN];
};
-enum mmal_parameter_camera_info_flash_type_t {
+enum mmal_parameter_camera_info_flash_type {
/* Make values explicit to ensure they match values in config ini */
MMAL_PARAMETER_CAMERA_INFO_FLASH_TYPE_XENON = 0,
MMAL_PARAMETER_CAMERA_INFO_FLASH_TYPE_LED = 1,
@@ -769,16 +769,16 @@ enum mmal_parameter_camera_info_flash_type_t {
MMAL_PARAMETER_CAMERA_INFO_FLASH_TYPE_MAX = 0x7FFFFFFF
};
-struct mmal_parameter_camera_info_flash_t {
- enum mmal_parameter_camera_info_flash_type_t flash_type;
+struct mmal_parameter_camera_info_flash {
+ enum mmal_parameter_camera_info_flash_type flash_type;
};
-struct mmal_parameter_camera_info_t {
- u32 num_cameras;
- u32 num_flashes;
- struct mmal_parameter_camera_info_camera_t
- cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
- struct mmal_parameter_camera_info_flash_t
+struct mmal_parameter_camera_info {
+ u32 num_cameras;
+ u32 num_flashes;
+ struct mmal_parameter_camera_info_camera
+ cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
+ struct mmal_parameter_camera_info_flash
flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
};
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 25/34] staging: vchiq_arm: Register vcsm-cma as a platform driver
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (23 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 24/34] staging: mmal-vchiq: Fix formatting errors in mmal_parameters.h Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 9:26 ` [PATCH v2 26/34] staging: vchiq_arm: Set up dma ranges on child devices Laurent Pinchart
` (9 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.com>
Following the same pattern as bcm2835-camera and bcm2835-audio,
register the vcsm-cma driver as a platform driver
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index a1ea9777a444..c5c7af28c1c8 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -107,6 +107,7 @@ static struct class *vchiq_class;
static DEFINE_SPINLOCK(msg_queue_spinlock);
static struct platform_device *bcm2835_camera;
static struct platform_device *bcm2835_audio;
+static struct platform_device *vcsm_cma;
static struct vchiq_drvdata bcm2835_drvdata = {
.cache_line_size = 32,
@@ -2789,6 +2790,7 @@ static int vchiq_probe(struct platform_device *pdev)
VCHIQ_VERSION, VCHIQ_VERSION_MIN,
MAJOR(vchiq_devid), MINOR(vchiq_devid));
+ vcsm_cma = vchiq_register_child(pdev, "vcsm-cma");
bcm2835_camera = vchiq_register_child(pdev, "bcm2835-camera");
bcm2835_audio = vchiq_register_child(pdev, "bcm2835_audio");
@@ -2804,6 +2806,7 @@ static int vchiq_probe(struct platform_device *pdev)
static int vchiq_remove(struct platform_device *pdev)
{
platform_device_unregister(bcm2835_camera);
+ platform_device_unregister(vcsm_cma);
vchiq_debugfs_deinit();
device_destroy(vchiq_class, vchiq_devid);
cdev_del(&vchiq_cdev);
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 26/34] staging: vchiq_arm: Set up dma ranges on child devices
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (24 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 25/34] staging: vchiq_arm: Register vcsm-cma as a platform driver Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 16:54 ` Nicolas Saenz Julienne
2020-05-04 9:26 ` [PATCH v2 27/34] staging: vchiq: Use the old dma controller for OF config on platform devices Laurent Pinchart
` (8 subsequent siblings)
34 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
The VCHIQ driver now loads the audio, camera, codec, and vc-sm
drivers as platform drivers. However they were not being given
the correct DMA configuration.
Call of_dma_configure with the parent (VCHIQ) parameters to be
inherited by the child.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index c5c7af28c1c8..15ccd624aaab 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -2733,6 +2733,12 @@ vchiq_register_child(struct platform_device *pdev, const char *name)
child = NULL;
}
+ /*
+ * We want the dma-ranges etc to be copied from the parent VCHIQ device
+ * to be passed on to the children too.
+ */
+ of_dma_configure(&new_dev->dev, pdev->dev.of_node, true);
+
return child;
}
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 26/34] staging: vchiq_arm: Set up dma ranges on child devices
2020-05-04 9:26 ` [PATCH v2 26/34] staging: vchiq_arm: Set up dma ranges on child devices Laurent Pinchart
@ 2020-05-04 16:54 ` Nicolas Saenz Julienne
2020-08-25 16:57 ` Jacopo Mondi
0 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-04 16:54 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]
On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> The VCHIQ driver now loads the audio, camera, codec, and vc-sm
> drivers as platform drivers. However they were not being given
> the correct DMA configuration.
>
> Call of_dma_configure with the parent (VCHIQ) parameters to be
> inherited by the child.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
> .../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 6 ++++++
> 1 file changed, 6 insertions(+)
>
> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> index c5c7af28c1c8..15ccd624aaab 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> @@ -2733,6 +2733,12 @@ vchiq_register_child(struct platform_device *pdev,
> const char *name)
> child = NULL;
> }
>
> + /*
> + * We want the dma-ranges etc to be copied from the parent VCHIQ device
> + * to be passed on to the children too.
> + */
> + of_dma_configure(&new_dev->dev, pdev->dev.of_node, true);
I think you could use struct platform_device_info's of_node_reused. See patch
908f6fc3a1405 ('usb: musb: sunxi: propagate devicetree node to glue pdev') for
an example. AFAIK of_dma_configure() is only to be used in bus code, and there
has been a huge effort in the past to make sure it says as such. With a proper
fwnode set of_dma_configure() will be called during the device's probe.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 26/34] staging: vchiq_arm: Set up dma ranges on child devices
2020-05-04 16:54 ` Nicolas Saenz Julienne
@ 2020-08-25 16:57 ` Jacopo Mondi
0 siblings, 0 replies; 111+ messages in thread
From: Jacopo Mondi @ 2020-08-25 16:57 UTC (permalink / raw)
To: Nicolas Saenz Julienne, phil
Cc: Laurent Pinchart, linux-media, Kieran Bingham,
Niklas Söderlund, Naushir Patuck, Dave Stevenson,
Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 3011 bytes --]
Hi Nicolas, Phil,
I'm in the process of sending a v2 of this series, trying to
reduce the patch count by only picking what's required to support
the ISP.
This patch and [30/34] puzzles me a bit, so I'll ask a few questions.
On Mon, May 04, 2020 at 06:54:32PM +0200, Nicolas Saenz Julienne wrote:
> On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> > From: Dave Stevenson <dave.stevenson@raspberrypi.org>
> >
> > The VCHIQ driver now loads the audio, camera, codec, and vc-sm
> > drivers as platform drivers. However they were not being given
> > the correct DMA configuration.
> >
> > Call of_dma_configure with the parent (VCHIQ) parameters to be
> > inherited by the child.
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> > .../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 6 ++++++
> > 1 file changed, 6 insertions(+)
> >
> > diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > index c5c7af28c1c8..15ccd624aaab 100644
> > --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > @@ -2733,6 +2733,12 @@ vchiq_register_child(struct platform_device *pdev,
> > const char *name)
> > child = NULL;
> > }
> >
> > + /*
> > + * We want the dma-ranges etc to be copied from the parent VCHIQ device
> > + * to be passed on to the children too.
> > + */
> > + of_dma_configure(&new_dev->dev, pdev->dev.of_node, true);
>
> I think you could use struct platform_device_info's of_node_reused. See patch
> 908f6fc3a1405 ('usb: musb: sunxi: propagate devicetree node to glue pdev') for
> an example. AFAIK of_dma_configure() is only to be used in bus code, and there
> has been a huge effort in the past to make sure it says as such. With a proper
> fwnode set of_dma_configure() will be called during the device's probe.
I just tried giving to the children devices the parent's fwnode and set
of_node_reused as done in 908f6fc3a1405 to avoid usage of
of_dma_configure, and now I have the parent being probed 4 times, one per
each children :) I guess that was not the expected result...
What is in your opinions the best course of actions for this and patch
[30/34] ? Can we live with 'of_dma_configure()' ? [1]
I am assuming currently the vchiq children are manually registered as
there are no entries in the DTS for them, so I'm kind of missing the
point of having [30/34] in. What am I missing ?
Thanks
j
[1] There are a few things that are not clear to me though, possible due
to my rather poor understanding of this core parts. In particular, as
the dma-ranges are assigned to the parent 'soc' node, and I assume
propagated to the children such as the 'vchiq' node, isnt' setting the
'parent' field of each newly created platform device enough to have
the property propagated ?
>
> Regards,
> Nicolas
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 27/34] staging: vchiq: Use the old dma controller for OF config on platform devices
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (25 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 26/34] staging: vchiq_arm: Set up dma ranges on child devices Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 15:44 ` Nicolas Saenz Julienne
2020-05-04 9:26 ` [PATCH v2 28/34] staging: vchiq_2835_arm: Implement a DMA pool for small bulk transfers Laurent Pinchart
` (7 subsequent siblings)
34 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
vchiq on Pi4 is no longer under the soc node, therefore it
doesn't get the dma-ranges for the VPU.
Switch to using the configuration of the old dma controller as
that will set the dma-ranges correctly.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../interface/vchiq_arm/vchiq_arm.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index 15ccd624aaab..d1a556f16499 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -2719,6 +2719,7 @@ vchiq_register_child(struct platform_device *pdev, const char *name)
{
struct platform_device_info pdevinfo;
struct platform_device *child;
+ struct device_node *np;
memset(&pdevinfo, 0, sizeof(pdevinfo));
@@ -2734,10 +2735,20 @@ vchiq_register_child(struct platform_device *pdev, const char *name)
}
/*
- * We want the dma-ranges etc to be copied from the parent VCHIQ device
- * to be passed on to the children too.
+ * We want the dma-ranges etc to be copied from a device with the
+ * correct dma-ranges for the VPU.
+ * VCHIQ on Pi4 is now under scb which doesn't get those dma-ranges.
+ * Take the "dma" node as going to be suitable as it sees the world
+ * through the same eyes as the VPU.
*/
- of_dma_configure(&new_dev->dev, pdev->dev.of_node, true);
+ np = of_find_node_by_path("dma");
+ if (!np)
+ np = pdev->dev.of_node;
+
+ of_dma_configure(&child->dev, np, true);
+
+ if (np != pdev->dev.of_node)
+ of_node_put(np);
return child;
}
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 27/34] staging: vchiq: Use the old dma controller for OF config on platform devices
2020-05-04 9:26 ` [PATCH v2 27/34] staging: vchiq: Use the old dma controller for OF config on platform devices Laurent Pinchart
@ 2020-05-04 15:44 ` Nicolas Saenz Julienne
0 siblings, 0 replies; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-04 15:44 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 2712 bytes --]
Hi Laurent, Dave,
On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> vchiq on Pi4 is no longer under the soc node, therefore it
> doesn't get the dma-ranges for the VPU.
This is not true in upstream's device-tree, and AFAIK, from a HW perspective,
the VPU interface didn't change interconnects, it's still connected to /soc
(highlighted by the fact that it has its DMA limitations).
Why did you found the need to move vchiq into /scb downstream?
> Switch to using the configuration of the old dma controller as
> that will set the dma-ranges correctly.
Ultimately, this is no different than copying DMA constraints from, say, the
I2C bus controller. The fact that this is a DMA controller has nothing to with
VPU's view of the system's memory (which dma-ranges symbolizes).
BTW, Laurent, I learned all about DMA trough one of your talks, it really
helped to set everything in place at the time :).
Regards,
Nicolas
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
> .../interface/vchiq_arm/vchiq_arm.c | 17 ++++++++++++++---
> 1 file changed, 14 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> index 15ccd624aaab..d1a556f16499 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> @@ -2719,6 +2719,7 @@ vchiq_register_child(struct platform_device *pdev, const
> char *name)
> {
> struct platform_device_info pdevinfo;
> struct platform_device *child;
> + struct device_node *np;
>
> memset(&pdevinfo, 0, sizeof(pdevinfo));
>
> @@ -2734,10 +2735,20 @@ vchiq_register_child(struct platform_device *pdev,
> const char *name)
> }
>
> /*
> - * We want the dma-ranges etc to be copied from the parent VCHIQ device
> - * to be passed on to the children too.
> + * We want the dma-ranges etc to be copied from a device with the
> + * correct dma-ranges for the VPU.
> + * VCHIQ on Pi4 is now under scb which doesn't get those dma-ranges.
> + * Take the "dma" node as going to be suitable as it sees the world
> + * through the same eyes as the VPU.
> */
> - of_dma_configure(&new_dev->dev, pdev->dev.of_node, true);
> + np = of_find_node_by_path("dma");
> + if (!np)
> + np = pdev->dev.of_node;
> +
> + of_dma_configure(&child->dev, np, true);
> +
> + if (np != pdev->dev.of_node)
> + of_node_put(np);
>
> return child;
> }
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 28/34] staging: vchiq_2835_arm: Implement a DMA pool for small bulk transfers
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (26 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 27/34] staging: vchiq: Use the old dma controller for OF config on platform devices Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 9:26 ` [PATCH v2 29/34] staging: vchiq: Add 36-bit address support Laurent Pinchart
` (6 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Oliver Gjoneski
From: Oliver Gjoneski <ogjoneski@gmail.com>
During a bulk transfer we request a DMA allocation to hold the
scatter-gather list. Most of the time, this allocation is small
(<< PAGE_SIZE), however it can be requested at a high enough frequency
to cause fragmentation and/or stress the CMA allocator (think time
spent in compaction here, or during allocations elsewhere).
Implement a pool to serve up small DMA allocations, falling back
to a coherent allocation if the request is greater than
VCHIQ_DMA_POOL_SIZE.
Signed-off-by: Oliver Gjoneski <ogjoneski@gmail.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../interface/vchiq_arm/vchiq_2835_arm.c | 38 ++++++++++++++++---
1 file changed, 33 insertions(+), 5 deletions(-)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
index c18c6ca0b6c0..3e422a7eb3f1 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
@@ -7,6 +7,7 @@
#include <linux/interrupt.h>
#include <linux/pagemap.h>
#include <linux/dma-mapping.h>
+#include <linux/dmapool.h>
#include <linux/io.h>
#include <linux/platform_device.h>
#include <linux/uaccess.h>
@@ -28,6 +29,8 @@
#define BELL0 0x00
#define BELL2 0x08
+#define VCHIQ_DMA_POOL_SIZE PAGE_SIZE
+
struct vchiq_2835_state {
int inited;
struct vchiq_arm_state arm_state;
@@ -37,6 +40,7 @@ struct vchiq_pagelist_info {
struct pagelist *pagelist;
size_t pagelist_buffer_size;
dma_addr_t dma_addr;
+ bool is_from_pool;
enum dma_data_direction dma_dir;
unsigned int num_pages;
unsigned int pages_need_release;
@@ -57,6 +61,7 @@ static void __iomem *g_regs;
* of 32.
*/
static unsigned int g_cache_line_size = 32;
+static struct dma_pool *g_dma_pool;
static unsigned int g_fragments_size;
static char *g_fragments_base;
static char *g_free_fragments;
@@ -159,6 +164,14 @@ int vchiq_platform_init(struct platform_device *pdev, struct vchiq_state *state)
}
g_dev = dev;
+ g_dma_pool = dmam_pool_create("vchiq_scatter_pool", dev,
+ VCHIQ_DMA_POOL_SIZE, g_cache_line_size,
+ 0);
+ if (!g_dma_pool) {
+ dev_err(dev, "failed to create dma pool");
+ return -ENOMEM;
+ }
+
vchiq_log_info(vchiq_arm_log_level,
"vchiq_init - done (slots %pK, phys %pad)",
vchiq_slot_zero, &slot_phys);
@@ -293,9 +306,14 @@ cleanup_pagelistinfo(struct vchiq_pagelist_info *pagelistinfo)
for (i = 0; i < pagelistinfo->num_pages; i++)
put_page(pagelistinfo->pages[i]);
}
-
- dma_free_coherent(g_dev, pagelistinfo->pagelist_buffer_size,
- pagelistinfo->pagelist, pagelistinfo->dma_addr);
+ if (pagelistinfo->is_from_pool) {
+ dma_pool_free(g_dma_pool, pagelistinfo->pagelist,
+ pagelistinfo->dma_addr);
+ } else {
+ dma_free_coherent(g_dev, pagelistinfo->pagelist_buffer_size,
+ pagelistinfo->pagelist,
+ pagelistinfo->dma_addr);
+ }
}
/* There is a potential problem with partial cache lines (pages?)
@@ -315,6 +333,7 @@ create_pagelist(char __user *buf, size_t count, unsigned short type)
u32 *addrs;
unsigned int num_pages, offset, i, k;
int actual_pages;
+ bool is_from_pool;
size_t pagelist_size;
struct scatterlist *scatterlist, *sg;
int dma_buffers;
@@ -341,8 +360,16 @@ create_pagelist(char __user *buf, size_t count, unsigned short type)
/* Allocate enough storage to hold the page pointers and the page
* list
*/
- pagelist = dma_alloc_coherent(g_dev, pagelist_size, &dma_addr,
- GFP_KERNEL);
+ if (pagelist_size > VCHIQ_DMA_POOL_SIZE) {
+ pagelist = dma_alloc_coherent(g_dev,
+ pagelist_size,
+ &dma_addr,
+ GFP_KERNEL);
+ is_from_pool = false;
+ } else {
+ pagelist = dma_pool_alloc(g_dma_pool, GFP_KERNEL, &dma_addr);
+ is_from_pool = true;
+ }
vchiq_log_trace(vchiq_arm_log_level, "%s - %pK", __func__, pagelist);
@@ -363,6 +390,7 @@ create_pagelist(char __user *buf, size_t count, unsigned short type)
pagelistinfo->pagelist = pagelist;
pagelistinfo->pagelist_buffer_size = pagelist_size;
pagelistinfo->dma_addr = dma_addr;
+ pagelistinfo->is_from_pool = is_from_pool;
pagelistinfo->dma_dir = (type == PAGELIST_WRITE) ?
DMA_TO_DEVICE : DMA_FROM_DEVICE;
pagelistinfo->num_pages = num_pages;
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 29/34] staging: vchiq: Add 36-bit address support
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (27 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 28/34] staging: vchiq_2835_arm: Implement a DMA pool for small bulk transfers Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 17:40 ` Nicolas Saenz Julienne
2020-05-05 13:22 ` kbuild test robot
2020-05-04 9:26 ` [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes Laurent Pinchart
` (5 subsequent siblings)
34 siblings, 2 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
From: Phil Elwell <phil@raspberrypi.org>
Conditional on a new compatible string, change the pagelist encoding
such that the top 24 bits are the pfn, leaving 8 bits for run length
(-1).
Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../interface/vchiq_arm/vchiq_2835_arm.c | 88 ++++++++++++++-----
.../interface/vchiq_arm/vchiq_arm.c | 6 ++
.../interface/vchiq_arm/vchiq_arm.h | 1 +
3 files changed, 74 insertions(+), 21 deletions(-)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
index 3e422a7eb3f1..ecec84ad4345 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
@@ -16,6 +16,8 @@
#include <soc/bcm2835/raspberrypi-firmware.h>
#define TOTAL_SLOTS (VCHIQ_SLOT_ZERO_SLOTS + 2 * 32)
+#define VC_SAFE(x) (g_use_36bit_addrs ? ((u32)(x) | 0xc0000000) : (u32)(x))
+#define IS_VC_SAFE(x) (g_use_36bit_addrs ? !((x) & ~0x3fffffffull) : 1)
#include "vchiq_arm.h"
#include "vchiq_connected.h"
@@ -62,6 +64,7 @@ static void __iomem *g_regs;
*/
static unsigned int g_cache_line_size = 32;
static struct dma_pool *g_dma_pool;
+static unsigned int g_use_36bit_addrs = 0;
static unsigned int g_fragments_size;
static char *g_fragments_base;
static char *g_free_fragments;
@@ -104,6 +107,8 @@ int vchiq_platform_init(struct platform_device *pdev, struct vchiq_state *state)
g_cache_line_size = drvdata->cache_line_size;
g_fragments_size = 2 * g_cache_line_size;
+ g_use_36bit_addrs = (dev->dma_pfn_offset == 0);
+
/* Allocate space for the channels in coherent memory */
slot_mem_size = PAGE_ALIGN(TOTAL_SLOTS * VCHIQ_SLOT_SIZE);
frag_mem_size = PAGE_ALIGN(g_fragments_size * MAX_FRAGMENTS);
@@ -115,14 +120,21 @@ int vchiq_platform_init(struct platform_device *pdev, struct vchiq_state *state)
return -ENOMEM;
}
+ if (!IS_VC_SAFE(slot_phys)) {
+ dev_err(dev, "allocated DMA memory %pad is not VC-safe\n",
+ &slot_phys);
+ return -ENOMEM;
+ }
+
WARN_ON(((unsigned long)slot_mem & (PAGE_SIZE - 1)) != 0);
+ channelbase = VC_SAFE(slot_phys);
vchiq_slot_zero = vchiq_init_slots(slot_mem, slot_mem_size);
if (!vchiq_slot_zero)
return -EINVAL;
vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_OFFSET_IDX] =
- (int)slot_phys + slot_mem_size;
+ channelbase + slot_mem_size;
vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_COUNT_IDX] =
MAX_FRAGMENTS;
@@ -155,7 +167,6 @@ int vchiq_platform_init(struct platform_device *pdev, struct vchiq_state *state)
}
/* Send the base address of the slots to VideoCore */
- channelbase = slot_phys;
err = rpi_firmware_property(fw, RPI_FIRMWARE_VCHIQ_INIT,
&channelbase, sizeof(channelbase));
if (err || channelbase) {
@@ -241,7 +252,7 @@ vchiq_prepare_bulk_data(struct vchiq_bulk *bulk, void *offset, int size,
if (!pagelistinfo)
return VCHIQ_ERROR;
- bulk->data = (void *)(unsigned long)pagelistinfo->dma_addr;
+ bulk->data = (void *)VC_SAFE(pagelistinfo->dma_addr);
/*
* Store the pagelistinfo address in remote_data,
@@ -475,25 +486,60 @@ create_pagelist(char __user *buf, size_t count, unsigned short type)
/* Combine adjacent blocks for performance */
k = 0;
- for_each_sg(scatterlist, sg, dma_buffers, i) {
- u32 len = sg_dma_len(sg);
- u32 addr = sg_dma_address(sg);
+ if (g_use_36bit_addrs) {
+ for_each_sg(scatterlist, sg, dma_buffers, i) {
+ u32 len = sg_dma_len(sg);
+ u64 addr = sg_dma_address(sg);
+ u32 page_id = (u32)((addr >> 4) & ~0xff);
+ u32 sg_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;
- /* Note: addrs is the address + page_count - 1
- * The firmware expects blocks after the first to be page-
- * aligned and a multiple of the page size
- */
- WARN_ON(len == 0);
- WARN_ON(i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK));
- WARN_ON(i && (addr & ~PAGE_MASK));
- if (k > 0 &&
- ((addrs[k - 1] & PAGE_MASK) +
- (((addrs[k - 1] & ~PAGE_MASK) + 1) << PAGE_SHIFT))
- == (addr & PAGE_MASK))
- addrs[k - 1] += ((len + PAGE_SIZE - 1) >> PAGE_SHIFT);
- else
- addrs[k++] = (addr & PAGE_MASK) |
- (((len + PAGE_SIZE - 1) >> PAGE_SHIFT) - 1);
+ /* Note: addrs is the address + page_count - 1
+ * The firmware expects blocks after the first to be page-
+ * aligned and a multiple of the page size
+ */
+ WARN_ON(len == 0);
+ WARN_ON(i &&
+ (i != (dma_buffers - 1)) && (len & ~PAGE_MASK));
+ WARN_ON(i && (addr & ~PAGE_MASK));
+ WARN_ON(upper_32_bits(addr) > 0xf);
+ if (k > 0 &&
+ ((addrs[k - 1] & ~0xff) +
+ (((addrs[k - 1] & 0xff) + 1) << 8)
+ == page_id)) {
+ u32 inc_pages = min(sg_pages,
+ 0xff - (addrs[k - 1] & 0xff));
+ addrs[k - 1] += inc_pages;
+ page_id += inc_pages << 8;
+ sg_pages -= inc_pages;
+ }
+ while (sg_pages) {
+ u32 inc_pages = min(sg_pages, 0x100u);
+ addrs[k++] = page_id | (inc_pages - 1);
+ page_id += inc_pages << 8;
+ sg_pages -= inc_pages;
+ }
+ }
+ } else {
+ for_each_sg(scatterlist, sg, dma_buffers, i) {
+ u32 len = sg_dma_len(sg);
+ u32 addr = VC_SAFE(sg_dma_address(sg));
+ u32 new_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+ /* Note: addrs is the address + page_count - 1
+ * The firmware expects blocks after the first to be page-
+ * aligned and a multiple of the page size
+ */
+ WARN_ON(len == 0);
+ WARN_ON(i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK));
+ WARN_ON(i && (addr & ~PAGE_MASK));
+ if (k > 0 &&
+ ((addrs[k - 1] & PAGE_MASK) +
+ (((addrs[k - 1] & ~PAGE_MASK) + 1) << PAGE_SHIFT))
+ == (addr & PAGE_MASK))
+ addrs[k - 1] += new_pages;
+ else
+ addrs[k++] = (addr & PAGE_MASK) | (new_pages - 1);
+ }
}
/* Partial cache lines (fragments) require special measures */
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index d1a556f16499..dd3c8f829daa 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -117,6 +117,11 @@ static struct vchiq_drvdata bcm2836_drvdata = {
.cache_line_size = 64,
};
+static struct vchiq_drvdata bcm2711_drvdata = {
+ .cache_line_size = 64,
+ .use_36bit_addrs = true,
+};
+
static const char *const ioctl_names[] = {
"CONNECT",
"SHUTDOWN",
@@ -2710,6 +2715,7 @@ void vchiq_platform_conn_state_changed(struct vchiq_state *state,
static const struct of_device_id vchiq_of_match[] = {
{ .compatible = "brcm,bcm2835-vchiq", .data = &bcm2835_drvdata },
{ .compatible = "brcm,bcm2836-vchiq", .data = &bcm2836_drvdata },
+ { .compatible = "brcm,bcm2711-vchiq", .data = &bcm2711_drvdata },
{},
};
MODULE_DEVICE_TABLE(of, vchiq_of_match);
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h
index 0784c5002417..f8b1c005af62 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h
@@ -52,6 +52,7 @@ struct vchiq_arm_state {
struct vchiq_drvdata {
const unsigned int cache_line_size;
+ const bool use_36bit_addrs;
struct rpi_firmware *fw;
};
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 29/34] staging: vchiq: Add 36-bit address support
2020-05-04 9:26 ` [PATCH v2 29/34] staging: vchiq: Add 36-bit address support Laurent Pinchart
@ 2020-05-04 17:40 ` Nicolas Saenz Julienne
2020-05-04 20:46 ` Phil Elwell
2020-05-05 13:22 ` kbuild test robot
1 sibling, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-04 17:40 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
[-- Attachment #1: Type: text/plain, Size: 8579 bytes --]
Hi Phil, Laurent,
On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> From: Phil Elwell <phil@raspberrypi.org>
>
> Conditional on a new compatible string, change the pagelist encoding
> such that the top 24 bits are the pfn, leaving 8 bits for run length
> (-1).
>
> Signed-off-by: Phil Elwell <phil@raspberrypi.org>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
> .../interface/vchiq_arm/vchiq_2835_arm.c | 88 ++++++++++++++-----
> .../interface/vchiq_arm/vchiq_arm.c | 6 ++
> .../interface/vchiq_arm/vchiq_arm.h | 1 +
> 3 files changed, 74 insertions(+), 21 deletions(-)
>
> diff --git
> a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> index 3e422a7eb3f1..ecec84ad4345 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> @@ -16,6 +16,8 @@
> #include <soc/bcm2835/raspberrypi-firmware.h>
>
> #define TOTAL_SLOTS (VCHIQ_SLOT_ZERO_SLOTS + 2 * 32)
> +#define VC_SAFE(x) (g_use_36bit_addrs ? ((u32)(x) | 0xc0000000) : (u32)(x))
> +#define IS_VC_SAFE(x) (g_use_36bit_addrs ? !((x) & ~0x3fffffffull) : 1)
As I commented earlier, this is a sign your dma-ranges are wrong, most of the
code below reimplements what is already done by dma-direct (see
kernel/dma/direct.c). Once properly setup, you should be able to use whatever
phys address dmam_alloc_coherent() provides and drop g_use_36bit_addrs.
Note that on arm32+LPAE, dma-direct/swiotlb are the default dma_ops, so this
also applies there.
Regards,
Nicolas
> #include "vchiq_arm.h"
> #include "vchiq_connected.h"
> @@ -62,6 +64,7 @@ static void __iomem *g_regs;
> */
> static unsigned int g_cache_line_size = 32;
> static struct dma_pool *g_dma_pool;
> +static unsigned int g_use_36bit_addrs = 0;
> static unsigned int g_fragments_size;
> static char *g_fragments_base;
> static char *g_free_fragments;
> @@ -104,6 +107,8 @@ int vchiq_platform_init(struct platform_device *pdev,
> struct vchiq_state *state)
> g_cache_line_size = drvdata->cache_line_size;
> g_fragments_size = 2 * g_cache_line_size;
>
> + g_use_36bit_addrs = (dev->dma_pfn_offset == 0);
> +
> /* Allocate space for the channels in coherent memory */
> slot_mem_size = PAGE_ALIGN(TOTAL_SLOTS * VCHIQ_SLOT_SIZE);
> frag_mem_size = PAGE_ALIGN(g_fragments_size * MAX_FRAGMENTS);
> @@ -115,14 +120,21 @@ int vchiq_platform_init(struct platform_device *pdev,
> struct vchiq_state *state)
> return -ENOMEM;
> }
>
> + if (!IS_VC_SAFE(slot_phys)) {
> + dev_err(dev, "allocated DMA memory %pad is not VC-safe\n",
> + &slot_phys);
> + return -ENOMEM;
> + }
> +
> WARN_ON(((unsigned long)slot_mem & (PAGE_SIZE - 1)) != 0);
> + channelbase = VC_SAFE(slot_phys);
>
> vchiq_slot_zero = vchiq_init_slots(slot_mem, slot_mem_size);
> if (!vchiq_slot_zero)
> return -EINVAL;
>
> vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_OFFSET_IDX] =
> - (int)slot_phys + slot_mem_size;
> + channelbase + slot_mem_size;
> vchiq_slot_zero->platform_data[VCHIQ_PLATFORM_FRAGMENTS_COUNT_IDX] =
> MAX_FRAGMENTS;
>
> @@ -155,7 +167,6 @@ int vchiq_platform_init(struct platform_device *pdev,
> struct vchiq_state *state)
> }
>
> /* Send the base address of the slots to VideoCore */
> - channelbase = slot_phys;
> err = rpi_firmware_property(fw, RPI_FIRMWARE_VCHIQ_INIT,
> &channelbase, sizeof(channelbase));
> if (err || channelbase) {
> @@ -241,7 +252,7 @@ vchiq_prepare_bulk_data(struct vchiq_bulk *bulk, void
> *offset, int size,
> if (!pagelistinfo)
> return VCHIQ_ERROR;
>
> - bulk->data = (void *)(unsigned long)pagelistinfo->dma_addr;
> + bulk->data = (void *)VC_SAFE(pagelistinfo->dma_addr);
>
> /*
> * Store the pagelistinfo address in remote_data,
> @@ -475,25 +486,60 @@ create_pagelist(char __user *buf, size_t count, unsigned
> short type)
>
> /* Combine adjacent blocks for performance */
> k = 0;
> - for_each_sg(scatterlist, sg, dma_buffers, i) {
> - u32 len = sg_dma_len(sg);
> - u32 addr = sg_dma_address(sg);
> + if (g_use_36bit_addrs) {
> + for_each_sg(scatterlist, sg, dma_buffers, i) {
> + u32 len = sg_dma_len(sg);
> + u64 addr = sg_dma_address(sg);
> + u32 page_id = (u32)((addr >> 4) & ~0xff);
> + u32 sg_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;
>
> - /* Note: addrs is the address + page_count - 1
> - * The firmware expects blocks after the first to be page-
> - * aligned and a multiple of the page size
> - */
> - WARN_ON(len == 0);
> - WARN_ON(i && (i != (dma_buffers - 1)) && (len & ~PAGE_MASK));
> - WARN_ON(i && (addr & ~PAGE_MASK));
> - if (k > 0 &&
> - ((addrs[k - 1] & PAGE_MASK) +
> - (((addrs[k - 1] & ~PAGE_MASK) + 1) << PAGE_SHIFT))
> - == (addr & PAGE_MASK))
> - addrs[k - 1] += ((len + PAGE_SIZE - 1) >> PAGE_SHIFT);
> - else
> - addrs[k++] = (addr & PAGE_MASK) |
> - (((len + PAGE_SIZE - 1) >> PAGE_SHIFT) - 1);
> + /* Note: addrs is the address + page_count - 1
> + * The firmware expects blocks after the first to be
> page-
> + * aligned and a multiple of the page size
> + */
> + WARN_ON(len == 0);
> + WARN_ON(i &&
> + (i != (dma_buffers - 1)) && (len & ~PAGE_MASK));
> + WARN_ON(i && (addr & ~PAGE_MASK));
> + WARN_ON(upper_32_bits(addr) > 0xf);
> + if (k > 0 &&
> + ((addrs[k - 1] & ~0xff) +
> + (((addrs[k - 1] & 0xff) + 1) << 8)
> + == page_id)) {
> + u32 inc_pages = min(sg_pages,
> + 0xff - (addrs[k - 1] &
> 0xff));
> + addrs[k - 1] += inc_pages;
> + page_id += inc_pages << 8;
> + sg_pages -= inc_pages;
> + }
> + while (sg_pages) {
> + u32 inc_pages = min(sg_pages, 0x100u);
> + addrs[k++] = page_id | (inc_pages - 1);
> + page_id += inc_pages << 8;
> + sg_pages -= inc_pages;
> + }
> + }
> + } else {
> + for_each_sg(scatterlist, sg, dma_buffers, i) {
> + u32 len = sg_dma_len(sg);
> + u32 addr = VC_SAFE(sg_dma_address(sg));
> + u32 new_pages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;
> +
> + /* Note: addrs is the address + page_count - 1
> + * The firmware expects blocks after the first to be
> page-
> + * aligned and a multiple of the page size
> + */
> + WARN_ON(len == 0);
> + WARN_ON(i && (i != (dma_buffers - 1)) && (len &
> ~PAGE_MASK));
> + WARN_ON(i && (addr & ~PAGE_MASK));
> + if (k > 0 &&
> + ((addrs[k - 1] & PAGE_MASK) +
> + (((addrs[k - 1] & ~PAGE_MASK) + 1) << PAGE_SHIFT))
> + == (addr & PAGE_MASK))
> + addrs[k - 1] += new_pages;
> + else
> + addrs[k++] = (addr & PAGE_MASK) | (new_pages -
> 1);
> + }
> }
>
> /* Partial cache lines (fragments) require special measures */
> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> index d1a556f16499..dd3c8f829daa 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> @@ -117,6 +117,11 @@ static struct vchiq_drvdata bcm2836_drvdata = {
> .cache_line_size = 64,
> };
>
> +static struct vchiq_drvdata bcm2711_drvdata = {
> + .cache_line_size = 64,
> + .use_36bit_addrs = true,
> +};
> +
> static const char *const ioctl_names[] = {
> "CONNECT",
> "SHUTDOWN",
> @@ -2710,6 +2715,7 @@ void vchiq_platform_conn_state_changed(struct
> vchiq_state *state,
> static const struct of_device_id vchiq_of_match[] = {
> { .compatible = "brcm,bcm2835-vchiq", .data = &bcm2835_drvdata },
> { .compatible = "brcm,bcm2836-vchiq", .data = &bcm2836_drvdata },
> + { .compatible = "brcm,bcm2711-vchiq", .data = &bcm2711_drvdata },
> {},
> };
> MODULE_DEVICE_TABLE(of, vchiq_of_match);
> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h
> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h
> index 0784c5002417..f8b1c005af62 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.h
> @@ -52,6 +52,7 @@ struct vchiq_arm_state {
>
> struct vchiq_drvdata {
> const unsigned int cache_line_size;
> + const bool use_36bit_addrs;
> struct rpi_firmware *fw;
> };
>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 29/34] staging: vchiq: Add 36-bit address support
2020-05-04 17:40 ` Nicolas Saenz Julienne
@ 2020-05-04 20:46 ` Phil Elwell
2020-05-05 10:13 ` Nicolas Saenz Julienne
0 siblings, 1 reply; 111+ messages in thread
From: Phil Elwell @ 2020-05-04 20:46 UTC (permalink / raw)
To: Nicolas Saenz Julienne, Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
Hi Nicolas,
On 04/05/2020 18:40, Nicolas Saenz Julienne wrote:
> Hi Phil, Laurent,
>
> On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
>> From: Phil Elwell <phil@raspberrypi.org>
>>
>> Conditional on a new compatible string, change the pagelist encoding
>> such that the top 24 bits are the pfn, leaving 8 bits for run length
>> (-1).
>>
>> Signed-off-by: Phil Elwell <phil@raspberrypi.org>
>> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
>> ---
>> .../interface/vchiq_arm/vchiq_2835_arm.c | 88 ++++++++++++++-----
>> .../interface/vchiq_arm/vchiq_arm.c | 6 ++
>> .../interface/vchiq_arm/vchiq_arm.h | 1 +
>> 3 files changed, 74 insertions(+), 21 deletions(-)
>>
>> diff --git
>> a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
>> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
>> index 3e422a7eb3f1..ecec84ad4345 100644
>> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
>> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
>> @@ -16,6 +16,8 @@
>> #include <soc/bcm2835/raspberrypi-firmware.h>
>> #define TOTAL_SLOTS (VCHIQ_SLOT_ZERO_SLOTS + 2 * 32)
>> +#define VC_SAFE(x) (g_use_36bit_addrs ? ((u32)(x) | 0xc0000000) : (u32)(x))
>> +#define IS_VC_SAFE(x) (g_use_36bit_addrs ? !((x) & ~0x3fffffffull) : 1)
>
> As I commented earlier, this is a sign your dma-ranges are wrong, most of the
> code below reimplements what is already done by dma-direct (see
> kernel/dma/direct.c). Once properly setup, you should be able to use whatever
> phys address dmam_alloc_coherent() provides and drop g_use_36bit_addrs.
>
> Note that on arm32+LPAE, dma-direct/swiotlb are the default dma_ops, so this
> also applies there.
As I explained in an offline email, the problem is that VCHIQ needs access to two
kinds of "DMA" addresses - those suitable for the 40-bit DMA channel, and the
30-bit addresses that the VPU can use. Since each DT node only has access to a
single set of DMA ranges, I can't see how to use dma-direct to calculate addreses
for everything, but feel free to submit an alternative implementation showing how
it should be done.
Phil
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 29/34] staging: vchiq: Add 36-bit address support
2020-05-04 20:46 ` Phil Elwell
@ 2020-05-05 10:13 ` Nicolas Saenz Julienne
2020-05-05 10:57 ` Phil Elwell
0 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-05 10:13 UTC (permalink / raw)
To: Phil Elwell, Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
[-- Attachment #1: Type: text/plain, Size: 3027 bytes --]
On Mon, 2020-05-04 at 21:46 +0100, Phil Elwell wrote:
> Hi Nicolas,
>
> On 04/05/2020 18:40, Nicolas Saenz Julienne wrote:
> > Hi Phil, Laurent,
> >
> > On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> > > From: Phil Elwell <phil@raspberrypi.org>
> > >
> > > Conditional on a new compatible string, change the pagelist encoding
> > > such that the top 24 bits are the pfn, leaving 8 bits for run length
> > > (-1).
> > >
> > > Signed-off-by: Phil Elwell <phil@raspberrypi.org>
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> > > .../interface/vchiq_arm/vchiq_2835_arm.c | 88 ++++++++++++++-----
> > > .../interface/vchiq_arm/vchiq_arm.c | 6 ++
> > > .../interface/vchiq_arm/vchiq_arm.h | 1 +
> > > 3 files changed, 74 insertions(+), 21 deletions(-)
> > >
> > > diff --git
> > > a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > index 3e422a7eb3f1..ecec84ad4345 100644
> > > --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > @@ -16,6 +16,8 @@
> > > #include <soc/bcm2835/raspberrypi-firmware.h>
> > > #define TOTAL_SLOTS (VCHIQ_SLOT_ZERO_SLOTS + 2 * 32)
> > > +#define VC_SAFE(x) (g_use_36bit_addrs ? ((u32)(x) | 0xc0000000) :
> > > (u32)(x))
> > > +#define IS_VC_SAFE(x) (g_use_36bit_addrs ? !((x) & ~0x3fffffffull) : 1)
> >
> > As I commented earlier, this is a sign your dma-ranges are wrong, most of
> > the
> > code below reimplements what is already done by dma-direct (see
> > kernel/dma/direct.c). Once properly setup, you should be able to use
> > whatever
> > phys address dmam_alloc_coherent() provides and drop g_use_36bit_addrs.
> >
> > Note that on arm32+LPAE, dma-direct/swiotlb are the default dma_ops, so this
> > also applies there.
>
> As I explained in an offline email, the problem is that VCHIQ needs access to
We discussed this privately, but I wanted to start from scratch, specially as I
hope I won't be the only one reviewing this :).
> two
> kinds of "DMA" addresses - those suitable for the 40-bit DMA channel, and the
> 30-bit addresses that the VPU can use. Since each DT node only has access to a
> single set of DMA ranges, I can't see how to use dma-direct to calculate
> addreses
> for everything, but feel free to submit an alternative implementation showing
> how
> it should be done.
How about this):
- Move vchiq to /soc
- Get a handle to the relevant dma controller device pointer (so 30-bit DMA
controller on old RPis, 40-bit on RPi4)
- Allocate slotmem/pagelist with dma_alloc_coherent(vpu_dev, ...)
- Map pages with dma_map_sg(dma_dev, ...)
I hope I'm not missing anything, but short of implementing it and seeing the
end result, I think you'd be free of any rpi[1-3]/rpi4 distinction in the code.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 29/34] staging: vchiq: Add 36-bit address support
2020-05-05 10:13 ` Nicolas Saenz Julienne
@ 2020-05-05 10:57 ` Phil Elwell
0 siblings, 0 replies; 111+ messages in thread
From: Phil Elwell @ 2020-05-05 10:57 UTC (permalink / raw)
To: Nicolas Saenz Julienne
Cc: Laurent Pinchart, linux-media, Kieran Bingham, Jacopo Mondi,
Niklas Söderlund, Naushir Patuck, Dave Stevenson,
Phil Elwell
Hi Nicolas,
On Tue, 5 May 2020 at 11:13, Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> On Mon, 2020-05-04 at 21:46 +0100, Phil Elwell wrote:
> > Hi Nicolas,
> >
> > On 04/05/2020 18:40, Nicolas Saenz Julienne wrote:
> > > Hi Phil, Laurent,
> > >
> > > On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> > > > From: Phil Elwell <phil@raspberrypi.org>
> > > >
> > > > Conditional on a new compatible string, change the pagelist encoding
> > > > such that the top 24 bits are the pfn, leaving 8 bits for run length
> > > > (-1).
> > > >
> > > > Signed-off-by: Phil Elwell <phil@raspberrypi.org>
> > > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > > ---
> > > > .../interface/vchiq_arm/vchiq_2835_arm.c | 88 ++++++++++++++-----
> > > > .../interface/vchiq_arm/vchiq_arm.c | 6 ++
> > > > .../interface/vchiq_arm/vchiq_arm.h | 1 +
> > > > 3 files changed, 74 insertions(+), 21 deletions(-)
> > > >
> > > > diff --git
> > > > a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > > b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > > index 3e422a7eb3f1..ecec84ad4345 100644
> > > > --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > > +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
> > > > @@ -16,6 +16,8 @@
> > > > #include <soc/bcm2835/raspberrypi-firmware.h>
> > > > #define TOTAL_SLOTS (VCHIQ_SLOT_ZERO_SLOTS + 2 * 32)
> > > > +#define VC_SAFE(x) (g_use_36bit_addrs ? ((u32)(x) | 0xc0000000) :
> > > > (u32)(x))
> > > > +#define IS_VC_SAFE(x) (g_use_36bit_addrs ? !((x) & ~0x3fffffffull) : 1)
> > >
> > > As I commented earlier, this is a sign your dma-ranges are wrong, most of
> > > the
> > > code below reimplements what is already done by dma-direct (see
> > > kernel/dma/direct.c). Once properly setup, you should be able to use
> > > whatever
> > > phys address dmam_alloc_coherent() provides and drop g_use_36bit_addrs.
> > >
> > > Note that on arm32+LPAE, dma-direct/swiotlb are the default dma_ops, so this
> > > also applies there.
> >
> > As I explained in an offline email, the problem is that VCHIQ needs access to
>
> We discussed this privately, but I wanted to start from scratch, specially as I
> hope I won't be the only one reviewing this :).
>
> > two
> > kinds of "DMA" addresses - those suitable for the 40-bit DMA channel, and the
> > 30-bit addresses that the VPU can use. Since each DT node only has access to a
> > single set of DMA ranges, I can't see how to use dma-direct to calculate
> > addreses
> > for everything, but feel free to submit an alternative implementation showing
> > how
> > it should be done.
>
> How about this):
> - Move vchiq to /soc
> - Get a handle to the relevant dma controller device pointer (so 30-bit DMA
> controller on old RPis, 40-bit on RPi4)
> - Allocate slotmem/pagelist with dma_alloc_coherent(vpu_dev, ...)
> - Map pages with dma_map_sg(dma_dev, ...)
>
> I hope I'm not missing anything, but short of implementing it and seeing the
> end result, I think you'd be free of any rpi[1-3]/rpi4 distinction in the code.
Thanks for the suggestion - I hadn't considered using a device pointer
for the controller rather than the current (client) device. If that
works then I propose
to make the 40-bit dma property optional such that dma_dev defaults to
vpu_dev.
Phil
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 29/34] staging: vchiq: Add 36-bit address support
2020-05-04 9:26 ` [PATCH v2 29/34] staging: vchiq: Add 36-bit address support Laurent Pinchart
@ 2020-05-05 13:22 ` kbuild test robot
2020-05-05 13:22 ` kbuild test robot
1 sibling, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 13:22 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: kbuild-all, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
[-- Attachment #1: Type: text/plain, Size: 2531 bytes --]
Hi Laurent,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
config: s390-allyesconfig (attached as .config)
compiler: s390-linux-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=s390
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c: In function 'vchiq_prepare_bulk_data':
>> drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c:255:15: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
255 | bulk->data = (void *)VC_SAFE(pagelistinfo->dma_addr);
| ^
vim +255 drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
240
241 enum vchiq_status
242 vchiq_prepare_bulk_data(struct vchiq_bulk *bulk, void *offset, int size,
243 int dir)
244 {
245 struct vchiq_pagelist_info *pagelistinfo;
246
247 pagelistinfo = create_pagelist((char __user *)offset, size,
248 (dir == VCHIQ_BULK_RECEIVE)
249 ? PAGELIST_READ
250 : PAGELIST_WRITE);
251
252 if (!pagelistinfo)
253 return VCHIQ_ERROR;
254
> 255 bulk->data = (void *)VC_SAFE(pagelistinfo->dma_addr);
256
257 /*
258 * Store the pagelistinfo address in remote_data,
259 * which isn't used by the slave.
260 */
261 bulk->remote_data = pagelistinfo;
262
263 return VCHIQ_SUCCESS;
264 }
265
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 59279 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 29/34] staging: vchiq: Add 36-bit address support
@ 2020-05-05 13:22 ` kbuild test robot
0 siblings, 0 replies; 111+ messages in thread
From: kbuild test robot @ 2020-05-05 13:22 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2596 bytes --]
Hi Laurent,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.7-rc4]
[cannot apply to linuxtv-media/master anholt/for-next next-20200505]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url: https://github.com/0day-ci/linux/commits/Laurent-Pinchart/Drivers-for-the-BCM283x-CSI-2-CCP2-receiver-and-ISP/20200505-054310
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 9851a0dee7c28514f149f7e4f60ec1b06286cc1b
config: s390-allyesconfig (attached as .config)
compiler: s390-linux-gcc (GCC) 9.3.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day GCC_VERSION=9.3.0 make.cross ARCH=s390
If you fix the issue, kindly add following tag as appropriate
Reported-by: kbuild test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c: In function 'vchiq_prepare_bulk_data':
>> drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c:255:15: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
255 | bulk->data = (void *)VC_SAFE(pagelistinfo->dma_addr);
| ^
vim +255 drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
240
241 enum vchiq_status
242 vchiq_prepare_bulk_data(struct vchiq_bulk *bulk, void *offset, int size,
243 int dir)
244 {
245 struct vchiq_pagelist_info *pagelistinfo;
246
247 pagelistinfo = create_pagelist((char __user *)offset, size,
248 (dir == VCHIQ_BULK_RECEIVE)
249 ? PAGELIST_READ
250 : PAGELIST_WRITE);
251
252 if (!pagelistinfo)
253 return VCHIQ_ERROR;
254
> 255 bulk->data = (void *)VC_SAFE(pagelistinfo->dma_addr);
256
257 /*
258 * Store the pagelistinfo address in remote_data,
259 * which isn't used by the slave.
260 */
261 bulk->remote_data = pagelistinfo;
262
263 return VCHIQ_SUCCESS;
264 }
265
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 59279 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (28 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 29/34] staging: vchiq: Add 36-bit address support Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 17:12 ` Nicolas Saenz Julienne
2020-05-04 9:26 ` [PATCH v2 31/34] staging: vchiq_arm: Add a matching unregister call Laurent Pinchart
` (4 subsequent siblings)
34 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
From: Phil Elwell <phil@raspberrypi.com>
vchiq kernel clients are now instantiated as platform drivers rather
than using DT, but the children of the vchiq interface may still
benefit from access to DT properties. Give them the option of a
a sub-node of the vchiq parent for configuration and to allow
them to be disabled.
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index dd3c8f829daa..2325ab825941 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -2734,12 +2734,20 @@ vchiq_register_child(struct platform_device *pdev, const char *name)
pdevinfo.id = PLATFORM_DEVID_NONE;
pdevinfo.dma_mask = DMA_BIT_MASK(32);
+ np = of_get_child_by_name(pdev->dev.of_node, name);
+
+ /* Skip the child if it is explicitly disabled */
+ if (np && !of_device_is_available(np))
+ return NULL;
+
child = platform_device_register_full(&pdevinfo);
if (IS_ERR(child)) {
dev_warn(&pdev->dev, "%s not registered\n", name);
child = NULL;
}
+ child->dev.of_node = np;
+
/*
* We want the dma-ranges etc to be copied from a device with the
* correct dma-ranges for the VPU.
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes
2020-05-04 9:26 ` [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes Laurent Pinchart
@ 2020-05-04 17:12 ` Nicolas Saenz Julienne
2020-05-04 19:42 ` Phil Elwell
0 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-04 17:12 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
[-- Attachment #1: Type: text/plain, Size: 1987 bytes --]
Hi Phil, Laurent,
On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> From: Phil Elwell <phil@raspberrypi.com>
>
> vchiq kernel clients are now instantiated as platform drivers rather
> than using DT, but the children of the vchiq interface may still
> benefit from access to DT properties. Give them the option of a
> a sub-node of the vchiq parent for configuration and to allow
> them to be disabled.
>
> Signed-off-by: Phil Elwell <phil@raspberrypi.com>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
> .../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> index dd3c8f829daa..2325ab825941 100644
> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> @@ -2734,12 +2734,20 @@ vchiq_register_child(struct platform_device *pdev,
> const char *name)
> pdevinfo.id = PLATFORM_DEVID_NONE;
> pdevinfo.dma_mask = DMA_BIT_MASK(32);
>
> + np = of_get_child_by_name(pdev->dev.of_node, name);
> +
> + /* Skip the child if it is explicitly disabled */
> + if (np && !of_device_is_available(np))
> + return NULL;
I think this is alright, although I'd reshufle the code a little so it looks
nicer:
+ /* Skip the child if it is explicitly disabled */
+ np = of_get_child_by_name(pdev->dev.of_node, name);
+ if (np && !of_device_is_available(np))
+ return NULL;
> child = platform_device_register_full(&pdevinfo);
> if (IS_ERR(child)) {
> dev_warn(&pdev->dev, "%s not registered\n", name);
> child = NULL;
> }
>
> + child->dev.of_node = np;
Is this really needed? I'd rather have the parent's np (as commented in patch
26) as long as this is not a real device-tree defined platform device.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes
2020-05-04 17:12 ` Nicolas Saenz Julienne
@ 2020-05-04 19:42 ` Phil Elwell
2020-05-05 10:37 ` Nicolas Saenz Julienne
0 siblings, 1 reply; 111+ messages in thread
From: Phil Elwell @ 2020-05-04 19:42 UTC (permalink / raw)
To: Nicolas Saenz Julienne, Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Nicolas,
On 04/05/2020 18:12, Nicolas Saenz Julienne wrote:
> Hi Phil, Laurent,
>
> On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
>> From: Phil Elwell <phil@raspberrypi.com>
>>
>> vchiq kernel clients are now instantiated as platform drivers rather
>> than using DT, but the children of the vchiq interface may still
>> benefit from access to DT properties. Give them the option of a
>> a sub-node of the vchiq parent for configuration and to allow
>> them to be disabled.
>>
>> Signed-off-by: Phil Elwell <phil@raspberrypi.com>
>> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
>> ---
>> .../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 8 ++++++++
>> 1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
>> b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
>> index dd3c8f829daa..2325ab825941 100644
>> --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
>> +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
>> @@ -2734,12 +2734,20 @@ vchiq_register_child(struct platform_device *pdev,
>> const char *name)
>> pdevinfo.id = PLATFORM_DEVID_NONE;
>> pdevinfo.dma_mask = DMA_BIT_MASK(32);
>>
>> + np = of_get_child_by_name(pdev->dev.of_node, name);
>> +
>> + /* Skip the child if it is explicitly disabled */
>> + if (np && !of_device_is_available(np))
>> + return NULL;
>
> I think this is alright, although I'd reshufle the code a little so it looks
> nicer:
>
> + /* Skip the child if it is explicitly disabled */
> + np = of_get_child_by_name(pdev->dev.of_node, name);
> + if (np && !of_device_is_available(np))
> + return NULL;
I prefer the original.
>> child = platform_device_register_full(&pdevinfo);
>> if (IS_ERR(child)) {
>> dev_warn(&pdev->dev, "%s not registered\n", name);
>> child = NULL;
>> }
>>
>> + child->dev.of_node = np;
>
> Is this really needed? I'd rather have the parent's np (as commented in patch
> 26) as long as this is not a real device-tree defined platform device.
Unless the of_node pointer refers to the sub-node for the child, all children
would have to share a common set of properties, rather defeating the point of the
change.
Phil
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes
2020-05-04 19:42 ` Phil Elwell
@ 2020-05-05 10:37 ` Nicolas Saenz Julienne
2020-05-05 10:50 ` Phil Elwell
0 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-05 10:37 UTC (permalink / raw)
To: Phil Elwell, Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 3340 bytes --]
On Mon, 2020-05-04 at 20:42 +0100, Phil Elwell wrote:
> Hi Nicolas,
>
> On 04/05/2020 18:12, Nicolas Saenz Julienne wrote:
> > Hi Phil, Laurent,
> >
> > On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> > > From: Phil Elwell <phil@raspberrypi.com>
> > >
> > > vchiq kernel clients are now instantiated as platform drivers rather
> > > than using DT, but the children of the vchiq interface may still
> > > benefit from access to DT properties. Give them the option of a
> > > a sub-node of the vchiq parent for configuration and to allow
> > > them to be disabled.
> > >
> > > Signed-off-by: Phil Elwell <phil@raspberrypi.com>
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> > > .../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 8 ++++++++
> > > 1 file changed, 8 insertions(+)
> > >
> > > diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > index dd3c8f829daa..2325ab825941 100644
> > > --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > @@ -2734,12 +2734,20 @@ vchiq_register_child(struct platform_device *pdev,
> > > const char *name)
> > > pdevinfo.id = PLATFORM_DEVID_NONE;
> > > pdevinfo.dma_mask = DMA_BIT_MASK(32);
> > >
> > > + np = of_get_child_by_name(pdev->dev.of_node, name);
> > > +
> > > + /* Skip the child if it is explicitly disabled */
> > > + if (np && !of_device_is_available(np))
> > > + return NULL;
> >
> > I think this is alright, although I'd reshufle the code a little so it looks
> > nicer:
> >
> > + /* Skip the child if it is explicitly disabled */
> > + np = of_get_child_by_name(pdev->dev.of_node, name);
> > + if (np && !of_device_is_available(np))
> > + return NULL;
>
> I prefer the original.
Fair enough
> > > child = platform_device_register_full(&pdevinfo);
> > > if (IS_ERR(child)) {
> > > dev_warn(&pdev->dev, "%s not registered\n", name);
> > > child = NULL;
> > > }
> > >
> > > + child->dev.of_node = np;
> >
> > Is this really needed? I'd rather have the parent's np (as commented in
> > patch
> > 26) as long as this is not a real device-tree defined platform device.
>
> Unless the of_node pointer refers to the sub-node for the child, all children
> would have to share a common set of properties, rather defeating the point of
> the
> change.
Sorry I wasn't clear, my main point is that, since manually editing device
internals is bad a practice, specially after they have been registered (there
are potential races with dma_configure()/probe()). I want to make sure we need
it in the first place (i.e. I don't see any further usage of that device node).
If we can get rid of this line, we're better-off.
If we actually need the device node further down I propose two things:
- Use dev.of_node_reused, and do an children lookup anytime you need to get a
property. It's a one-liner in the end.
- Move device registration to DT. There has been some push-back of this in the
past, but IMO things like arm's SCMI already set a standard on what firmware
devices can do trough DT and it fits this situation.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes
2020-05-05 10:37 ` Nicolas Saenz Julienne
@ 2020-05-05 10:50 ` Phil Elwell
0 siblings, 0 replies; 111+ messages in thread
From: Phil Elwell @ 2020-05-05 10:50 UTC (permalink / raw)
To: Nicolas Saenz Julienne
Cc: Laurent Pinchart, linux-media, Kieran Bingham, Jacopo Mondi,
Niklas Söderlund, Naushir Patuck, Dave Stevenson
Hi Nicolas,
On Tue, 5 May 2020 at 11:37, Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> On Mon, 2020-05-04 at 20:42 +0100, Phil Elwell wrote:
> > Hi Nicolas,
> >
> > On 04/05/2020 18:12, Nicolas Saenz Julienne wrote:
> > > Hi Phil, Laurent,
> > >
> > > On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> > > > From: Phil Elwell <phil@raspberrypi.com>
> > > >
> > > > vchiq kernel clients are now instantiated as platform drivers rather
> > > > than using DT, but the children of the vchiq interface may still
> > > > benefit from access to DT properties. Give them the option of a
> > > > a sub-node of the vchiq parent for configuration and to allow
> > > > them to be disabled.
> > > >
> > > > Signed-off-by: Phil Elwell <phil@raspberrypi.com>
> > > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > > ---
> > > > .../staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 8 ++++++++
> > > > 1 file changed, 8 insertions(+)
> > > >
> > > > diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > > b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > > index dd3c8f829daa..2325ab825941 100644
> > > > --- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > > +++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> > > > @@ -2734,12 +2734,20 @@ vchiq_register_child(struct platform_device *pdev,
> > > > const char *name)
> > > > pdevinfo.id = PLATFORM_DEVID_NONE;
> > > > pdevinfo.dma_mask = DMA_BIT_MASK(32);
> > > >
> > > > + np = of_get_child_by_name(pdev->dev.of_node, name);
> > > > +
> > > > + /* Skip the child if it is explicitly disabled */
> > > > + if (np && !of_device_is_available(np))
> > > > + return NULL;
> > >
> > > I think this is alright, although I'd reshufle the code a little so it looks
> > > nicer:
> > >
> > > + /* Skip the child if it is explicitly disabled */
> > > + np = of_get_child_by_name(pdev->dev.of_node, name);
> > > + if (np && !of_device_is_available(np))
> > > + return NULL;
> >
> > I prefer the original.
>
> Fair enough
>
> > > > child = platform_device_register_full(&pdevinfo);
> > > > if (IS_ERR(child)) {
> > > > dev_warn(&pdev->dev, "%s not registered\n", name);
> > > > child = NULL;
> > > > }
> > > >
> > > > + child->dev.of_node = np;
> > >
> > > Is this really needed? I'd rather have the parent's np (as commented in
> > > patch
> > > 26) as long as this is not a real device-tree defined platform device.
> >
> > Unless the of_node pointer refers to the sub-node for the child, all children
> > would have to share a common set of properties, rather defeating the point of
> > the
> > change.
>
> Sorry I wasn't clear, my main point is that, since manually editing device
> internals is bad a practice, specially after they have been registered (there
> are potential races with dma_configure()/probe()). I want to make sure we need
> it in the first place (i.e. I don't see any further usage of that device node).
>
> If we can get rid of this line, we're better-off.
Thanks - that is much clearer.
> If we actually need the device node further down I propose two things:
> - Use dev.of_node_reused, and do an children lookup anytime you need to get a
> property. It's a one-liner in the end.
> - Move device registration to DT. There has been some push-back of this in the
> past, but IMO things like arm's SCMI already set a standard on what firmware
> devices can do trough DT and it fits this situation.
I much prefer registration via DT - enumerating the children in code rather than
data always felt like a baffling step backwards.
Phil
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 31/34] staging: vchiq_arm: Add a matching unregister call
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (29 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 30/34] staging: vchiq_arm: Give vchiq children DT nodes Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 9:26 ` [PATCH v2 32/34] media: videobuf2: Allow exporting of a struct dmabuf Laurent Pinchart
` (3 subsequent siblings)
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Phil Elwell
From: Phil Elwell <phil@raspberrypi.com>
All the registered children of vchiq have a corresponding call to
platform_device_unregister except bcm2835_audio. Fix that.
Fixes: 25c7597af20d ("staging: vchiq_arm: Register a platform device for audio")
Signed-off-by: Phil Elwell <phil@raspberrypi.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index 2325ab825941..59ae2d479ef4 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -2836,6 +2836,7 @@ static int vchiq_probe(struct platform_device *pdev)
static int vchiq_remove(struct platform_device *pdev)
{
+ platform_device_unregister(bcm2835_audio);
platform_device_unregister(bcm2835_camera);
platform_device_unregister(vcsm_cma);
vchiq_debugfs_deinit();
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* [PATCH v2 32/34] media: videobuf2: Allow exporting of a struct dmabuf
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (30 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 31/34] staging: vchiq_arm: Add a matching unregister call Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 13:36 ` Hans Verkuil
2020-05-04 9:26 ` [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP Laurent Pinchart
` (2 subsequent siblings)
34 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
From: Dave Stevenson <dave.stevenson@raspberrypi.org>
videobuf2 only allowed exporting a dmabuf as a file descriptor,
but there are instances where having the struct dma_buf is
useful within the kernel.
Split the current implementation into two, one step which
exports a struct dma_buf, and the second which converts that
into an fd.
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
---
.../media/common/videobuf2/videobuf2-core.c | 21 ++++++++++++++++---
include/media/videobuf2-core.h | 15 +++++++++++++
2 files changed, 33 insertions(+), 3 deletions(-)
diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
index 44d65f5be845..befdc89983e2 100644
--- a/drivers/media/common/videobuf2/videobuf2-core.c
+++ b/drivers/media/common/videobuf2/videobuf2-core.c
@@ -2073,12 +2073,12 @@ static int __find_plane_by_offset(struct vb2_queue *q, unsigned long off,
return -EINVAL;
}
-int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
- unsigned int index, unsigned int plane, unsigned int flags)
+int vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
+ unsigned int index, unsigned int plane,
+ unsigned int flags, struct dma_buf **dmabuf)
{
struct vb2_buffer *vb = NULL;
struct vb2_plane *vb_plane;
- int ret;
struct dma_buf *dbuf;
if (q->memory != VB2_MEMORY_MMAP) {
@@ -2128,6 +2128,21 @@ int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
return -EINVAL;
}
+ *dmabuf = dbuf;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(vb2_core_expbuf_dmabuf);
+
+int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
+ unsigned int index, unsigned int plane, unsigned int flags)
+{
+ struct dma_buf *dbuf;
+ int ret;
+
+ ret = vb2_core_expbuf_dmabuf(q, type, index, plane, flags, &dbuf);
+ if (ret)
+ return ret;
+
ret = dma_buf_fd(dbuf, flags & ~O_ACCMODE);
if (ret < 0) {
dprintk(3, "buffer %d, plane %d failed to export (%d)\n",
diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
index f11b96514cf7..86920d3264ab 100644
--- a/include/media/videobuf2-core.h
+++ b/include/media/videobuf2-core.h
@@ -875,6 +875,21 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type);
*/
int vb2_core_streamoff(struct vb2_queue *q, unsigned int type);
+/**
+ * vb2_core_expbuf_dmabuf() - Export a buffer as a dma_buf structure
+ * @q: videobuf2 queue
+ * @type: buffer type
+ * @index: id number of the buffer
+ * @plane: index of the plane to be exported, 0 for single plane queues
+ * @flags: flags for newly created file, currently only O_CLOEXEC is
+ * supported, refer to manual of open syscall for more details
+ * @dmabuf: Returns the dmabuf pointer
+ *
+ */
+int vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
+ unsigned int index, unsigned int plane,
+ unsigned int flags, struct dma_buf **dmabuf);
+
/**
* vb2_core_expbuf() - Export a buffer as a file descriptor.
* @q: pointer to &struct vb2_queue with videobuf2 queue.
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 32/34] media: videobuf2: Allow exporting of a struct dmabuf
2020-05-04 9:26 ` [PATCH v2 32/34] media: videobuf2: Allow exporting of a struct dmabuf Laurent Pinchart
@ 2020-05-04 13:36 ` Hans Verkuil
0 siblings, 0 replies; 111+ messages in thread
From: Hans Verkuil @ 2020-05-04 13:36 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson, Dave Stevenson
On 04/05/2020 11:26, Laurent Pinchart wrote:
> From: Dave Stevenson <dave.stevenson@raspberrypi.org>
>
> videobuf2 only allowed exporting a dmabuf as a file descriptor,
> but there are instances where having the struct dma_buf is
> useful within the kernel.
>
> Split the current implementation into two, one step which
> exports a struct dma_buf, and the second which converts that
> into an fd.
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.org>
> ---
> .../media/common/videobuf2/videobuf2-core.c | 21 ++++++++++++++++---
> include/media/videobuf2-core.h | 15 +++++++++++++
> 2 files changed, 33 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/media/common/videobuf2/videobuf2-core.c b/drivers/media/common/videobuf2/videobuf2-core.c
> index 44d65f5be845..befdc89983e2 100644
> --- a/drivers/media/common/videobuf2/videobuf2-core.c
> +++ b/drivers/media/common/videobuf2/videobuf2-core.c
> @@ -2073,12 +2073,12 @@ static int __find_plane_by_offset(struct vb2_queue *q, unsigned long off,
> return -EINVAL;
> }
>
> -int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
> - unsigned int index, unsigned int plane, unsigned int flags)
> +int vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
> + unsigned int index, unsigned int plane,
> + unsigned int flags, struct dma_buf **dmabuf)
Just returning struct dma_buf and using ERR_PTR() to report errors would
be much more elegant.
Otherwise this patch is fine.
Regards,
Hans
> {
> struct vb2_buffer *vb = NULL;
> struct vb2_plane *vb_plane;
> - int ret;
> struct dma_buf *dbuf;
>
> if (q->memory != VB2_MEMORY_MMAP) {
> @@ -2128,6 +2128,21 @@ int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
> return -EINVAL;
> }
>
> + *dmabuf = dbuf;
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(vb2_core_expbuf_dmabuf);
> +
> +int vb2_core_expbuf(struct vb2_queue *q, int *fd, unsigned int type,
> + unsigned int index, unsigned int plane, unsigned int flags)
> +{
> + struct dma_buf *dbuf;
> + int ret;
> +
> + ret = vb2_core_expbuf_dmabuf(q, type, index, plane, flags, &dbuf);
> + if (ret)
> + return ret;
> +
> ret = dma_buf_fd(dbuf, flags & ~O_ACCMODE);
> if (ret < 0) {
> dprintk(3, "buffer %d, plane %d failed to export (%d)\n",
> diff --git a/include/media/videobuf2-core.h b/include/media/videobuf2-core.h
> index f11b96514cf7..86920d3264ab 100644
> --- a/include/media/videobuf2-core.h
> +++ b/include/media/videobuf2-core.h
> @@ -875,6 +875,21 @@ int vb2_core_streamon(struct vb2_queue *q, unsigned int type);
> */
> int vb2_core_streamoff(struct vb2_queue *q, unsigned int type);
>
> +/**
> + * vb2_core_expbuf_dmabuf() - Export a buffer as a dma_buf structure
> + * @q: videobuf2 queue
> + * @type: buffer type
> + * @index: id number of the buffer
> + * @plane: index of the plane to be exported, 0 for single plane queues
> + * @flags: flags for newly created file, currently only O_CLOEXEC is
> + * supported, refer to manual of open syscall for more details
> + * @dmabuf: Returns the dmabuf pointer
> + *
> + */
> +int vb2_core_expbuf_dmabuf(struct vb2_queue *q, unsigned int type,
> + unsigned int index, unsigned int plane,
> + unsigned int flags, struct dma_buf **dmabuf);
> +
> /**
> * vb2_core_expbuf() - Export a buffer as a file descriptor.
> * @q: pointer to &struct vb2_queue with videobuf2 queue.
>
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (31 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 32/34] media: videobuf2: Allow exporting of a struct dmabuf Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-11 19:19 ` Nicolas Saenz Julienne
2020-05-18 12:02 ` Hans Verkuil
2020-05-04 9:26 ` [PATCH v2 34/34] staging: vchiq: Load bcm2835_isp driver from vchiq Laurent Pinchart
2020-05-04 15:15 ` [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Nicolas Saenz Julienne
34 siblings, 2 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Naushir Patuck <naush@raspberrypi.com>
Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
chips.
The driver interfaces though the VideoCore unit using the VCHIQ MMAL
interface.
ISP driver upported from from RaspberryPi BSP at revision:
6c3505be6c3e ("staging: vc04_services: isp: Make all references to bcm2835_isp_fmt const")
Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
[Adapt to staging by moving all modifications that in the BSP are scattered
in core components inside this directory]
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
.../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 41 +
drivers/staging/vc04_services/Kconfig | 2 +
drivers/staging/vc04_services/Makefile | 1 +
.../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
.../vc04_services/bcm2835-isp/Makefile | 10 +
.../bcm2835-isp/bcm2835-v4l2-isp.c | 1632 +++++++++++++++++
.../bcm2835-isp/bcm2835_isp_ctrls.h | 67 +
.../bcm2835-isp/bcm2835_isp_fmts.h | 301 +++
.../include/uapi/linux/bcm2835-isp.h | 333 ++++
.../staging/vc04_services/vchiq-mmal/Kconfig | 3 +-
.../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
.../vchiq-mmal/mmal-parameters.h | 153 +-
12 files changed, 2559 insertions(+), 2 deletions(-)
create mode 100644 drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
create mode 100644 drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
diff --git a/drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst b/drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
new file mode 100644
index 000000000000..f974774c8252
--- /dev/null
+++ b/drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
@@ -0,0 +1,41 @@
+.. Permission is granted to copy, distribute and/or modify this
+.. document under the terms of the GNU Free Documentation License,
+.. Version 1.1 or any later version published by the Free Software
+.. Foundation, with no Invariant Sections, no Front-Cover Texts
+.. and no Back-Cover Texts. A copy of the license is included at
+.. Documentation/media/uapi/fdl-appendix.rst.
+..
+.. TODO: replace it to GFDL-1.1-or-later WITH no-invariant-sections
+
+.. _v4l2-meta-fmt-bcm2835-isp-stats:
+
+*****************************************
+V4L2_META_FMT_BCM2835_ISP_STATS ('BSTA')
+*****************************************
+
+BCM2835 ISP Statistics
+
+Description
+===========
+
+The BCM2835 ISP hardware calculate image statistics for an input Bayer frame.
+These statistics are obtained from the "bcm2835-isp0-capture3" device node
+using the :c:type:`v4l2_meta_format` interface. They are formatted as described
+by the :c:type:`bcm2835_isp_stats` structure below.
+
+.. code-block:: c
+
+ #define DEFAULT_AWB_REGIONS_X 16
+ #define DEFAULT_AWB_REGIONS_Y 12
+
+ #define NUM_HISTOGRAMS 2
+ #define NUM_HISTOGRAM_BINS 128
+ #define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
+ #define FLOATING_REGIONS 16
+ #define AGC_REGIONS 16
+ #define FOCUS_REGIONS 12
+
+.. kernel-doc:: include/uapi/linux/bcm2835-isp.h
+ :functions: bcm2835_isp_stats_hist bcm2835_isp_stats_region
+ bcm2835_isp_stats_focus bcm2835_isp_stats
+
diff --git a/drivers/staging/vc04_services/Kconfig b/drivers/staging/vc04_services/Kconfig
index 7ae21d7f359b..f9fc88bd7343 100644
--- a/drivers/staging/vc04_services/Kconfig
+++ b/drivers/staging/vc04_services/Kconfig
@@ -23,6 +23,8 @@ source "drivers/staging/vc04_services/bcm2835-audio/Kconfig"
source "drivers/staging/vc04_services/bcm2835-camera/Kconfig"
+source "drivers/staging/vc04_services/bcm2835-isp/Kconfig"
+
source "drivers/staging/vc04_services/vc-sm-cma/Kconfig"
source "drivers/staging/vc04_services/vchiq-mmal/Kconfig"
diff --git a/drivers/staging/vc04_services/Makefile b/drivers/staging/vc04_services/Makefile
index 296aff05b9ab..5fc3b65224a3 100644
--- a/drivers/staging/vc04_services/Makefile
+++ b/drivers/staging/vc04_services/Makefile
@@ -12,6 +12,7 @@ vchiq-objs := \
obj-$(CONFIG_SND_BCM2835) += bcm2835-audio/
obj-$(CONFIG_VIDEO_BCM2835) += bcm2835-camera/
+obj-$(CONFIG_VIDEO_ISP_BCM2835) += bcm2835-isp/
obj-$(CONFIG_BCM_VC_SM_CMA) += vc-sm-cma/
obj-$(CONFIG_BCM2835_VCHIQ_MMAL) += vchiq-mmal/
diff --git a/drivers/staging/vc04_services/bcm2835-isp/Kconfig b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
new file mode 100644
index 000000000000..71b14acc297e
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/Kconfig
@@ -0,0 +1,14 @@
+config VIDEO_ISP_BCM2835
+ tristate "BCM2835 ISP support"
+ depends on MEDIA_SUPPORT
+ depends on VIDEO_V4L2 && (ARCH_BCM2835 || COMPILE_TEST)
+ depends on MEDIA_CONTROLLER
+ select BCM2835_VCHIQ_MMAL
+ select VIDEOBUF2_DMA_CONTIG
+ help
+ This is the V4L2 driver for the Broadcom BCM2835 ISP hardware.
+ This operates over the VCHIQ interface to a service running on
+ VideoCore.
+
+ To compile this driver as a module, choose M here: the module
+ will be called bcm2835-isp.
diff --git a/drivers/staging/vc04_services/bcm2835-isp/Makefile b/drivers/staging/vc04_services/bcm2835-isp/Makefile
new file mode 100644
index 000000000000..0120c55d7a31
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/Makefile
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+bcm2835-isp-objs := bcm2835-v4l2-isp.o
+
+obj-$(CONFIG_VIDEO_ISP_BCM2835) += bcm2835-isp.o
+
+ccflags-y += \
+ -I$(srctree)/$(src)/.. \
+ -I$(srctree)/$(src)/../include/uapi/ \
+ -I$(srctree)/$(src)/../vchiq-mmal/ \
+ -D__VCCOREVER__=0x04000000
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
new file mode 100644
index 000000000000..a32faab4b8dc
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
@@ -0,0 +1,1632 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Broadcom BCM2835 ISP driver
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
+#include <media/v4l2-event.h>
+#include <media/v4l2-ioctl.h>
+#include <media/videobuf2-dma-contig.h>
+
+#include "vchiq-mmal/mmal-msg.h"
+#include "vchiq-mmal/mmal-parameters.h"
+#include "vchiq-mmal/mmal-vchiq.h"
+
+#include "bcm2835_isp_ctrls.h"
+#include "bcm2835_isp_fmts.h"
+
+static unsigned int debug;
+module_param(debug, uint, 0644);
+MODULE_PARM_DESC(debug, "activates debug info");
+
+static unsigned int video_nr = 13;
+module_param(video_nr, uint, 0644);
+MODULE_PARM_DESC(video_nr, "base video device number");
+
+#define BCM2835_ISP_NAME "bcm2835-isp"
+#define BCM2835_ISP_ENTITY_NAME_LEN 32
+
+#define BCM2835_ISP_NUM_OUTPUTS 1
+#define BCM2835_ISP_NUM_CAPTURES 2
+#define BCM2835_ISP_NUM_METADATA 1
+
+#define BCM2835_ISP_NUM_NODES \
+ (BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES + \
+ BCM2835_ISP_NUM_METADATA)
+
+/* Default frame dimension of 1280 pixels. */
+#define DEFAULT_DIM 1280U
+/*
+ * Maximum frame dimension of 16384 pixels. Even though the ISP runs in tiles,
+ * have a sensible limit so that we do not create an excessive number of tiles
+ * to process.
+ */
+#define MAX_DIM 16384U
+/*
+ * Minimum frame dimension of 64 pixels. Anything lower, and the tiling
+ * algorihtm may not be able to cope when applying filter context.
+ */
+#define MIN_DIM 64U
+
+/* Per-queue, driver-specific private data */
+struct bcm2835_isp_q_data {
+ /*
+ * These parameters should be treated as gospel, with everything else
+ * being determined from them.
+ */
+ unsigned int bytesperline;
+ unsigned int width;
+ unsigned int height;
+ unsigned int sizeimage;
+ const struct bcm2835_isp_fmt *fmt;
+};
+
+/*
+ * Structure to describe a single node /dev/video<N> which represents a single
+ * input or output queue to the ISP device.
+ */
+struct bcm2835_isp_node {
+ int vfl_dir;
+ unsigned int id;
+ const char *name;
+ struct video_device vfd;
+ struct media_pad pad;
+ struct media_intf_devnode *intf_devnode;
+ struct media_link *intf_link;
+ struct mutex lock; /* top level device node lock */
+ struct mutex queue_lock;
+
+ struct vb2_queue queue;
+ unsigned int sequence;
+
+ /* The list of formats supported on the node. */
+ struct bcm2835_isp_fmt_list supported_fmts;
+
+ struct bcm2835_isp_q_data q_data;
+
+ /* Parent device structure */
+ struct bcm2835_isp_dev *dev;
+
+ bool registered;
+ bool media_node_registered;
+ bool queue_init;
+};
+
+/*
+ * Structure representing the entire ISP device, comprising several input and
+ * output nodes /dev/video<N>.
+ */
+struct bcm2835_isp_dev {
+ struct v4l2_device v4l2_dev;
+ struct device *dev;
+ struct v4l2_ctrl_handler ctrl_handler;
+ struct media_device mdev;
+ struct media_entity entity;
+ bool media_device_registered;
+ bool media_entity_registered;
+ struct vchiq_mmal_instance *mmal_instance;
+ struct vchiq_mmal_component *component;
+ struct completion frame_cmplt;
+
+ struct bcm2835_isp_node node[BCM2835_ISP_NUM_NODES];
+ struct media_pad pad[BCM2835_ISP_NUM_NODES];
+ atomic_t num_streaming;
+
+ /* Image pipeline controls. */
+ int r_gain;
+ int b_gain;
+};
+
+struct bcm2835_isp_buffer {
+ struct vb2_v4l2_buffer vb;
+ struct mmal_buffer mmal;
+};
+
+static
+inline struct bcm2835_isp_dev *node_get_dev(struct bcm2835_isp_node *node)
+{
+ return node->dev;
+}
+
+static inline bool node_is_output(struct bcm2835_isp_node *node)
+{
+ return node->queue.type == V4L2_BUF_TYPE_VIDEO_OUTPUT;
+}
+
+static inline bool node_is_capture(struct bcm2835_isp_node *node)
+{
+ return node->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE;
+}
+
+static inline bool node_is_stats(struct bcm2835_isp_node *node)
+{
+ return node->queue.type == V4L2_BUF_TYPE_META_CAPTURE;
+}
+
+static inline enum v4l2_buf_type index_to_queue_type(int index)
+{
+ if (index < BCM2835_ISP_NUM_OUTPUTS)
+ return V4L2_BUF_TYPE_VIDEO_OUTPUT;
+ else if (index < BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES)
+ return V4L2_BUF_TYPE_VIDEO_CAPTURE;
+ else
+ return V4L2_BUF_TYPE_META_CAPTURE;
+}
+
+static struct vchiq_mmal_port *get_port_data(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ if (!dev->component)
+ return NULL;
+
+ switch (node->queue.type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ return &dev->component->input[node->id];
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ case V4L2_BUF_TYPE_META_CAPTURE:
+ return &dev->component->output[node->id];
+ default:
+ v4l2_err(&dev->v4l2_dev, "%s: Invalid queue type %u\n",
+ __func__, node->queue.type);
+ break;
+ }
+ return NULL;
+}
+
+static int set_isp_param(struct bcm2835_isp_node *node, u32 parameter,
+ void *value, u32 value_size)
+{
+ struct vchiq_mmal_port *port = get_port_data(node);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
+ parameter, value, value_size);
+}
+
+static int set_wb_gains(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct mmal_parameter_awbgains gains = {
+ .r_gain = { dev->r_gain, 1000 },
+ .b_gain = { dev->b_gain, 1000 }
+ };
+
+ return set_isp_param(node, MMAL_PARAMETER_CUSTOM_AWB_GAINS,
+ &gains, sizeof(gains));
+}
+
+static int set_digital_gain(struct bcm2835_isp_node *node, uint32_t gain)
+{
+ struct mmal_parameter_rational digital_gain = {
+ .num = gain,
+ .den = 1000
+ };
+
+ return set_isp_param(node, MMAL_PARAMETER_DIGITAL_GAIN,
+ &digital_gain, sizeof(digital_gain));
+}
+
+static const struct bcm2835_isp_fmt *get_fmt(u32 mmal_fmt)
+{
+ unsigned int i;
+
+ for (i = 0; i < ARRAY_SIZE(supported_formats); i++) {
+ if (supported_formats[i].mmal_fmt == mmal_fmt)
+ return &supported_formats[i];
+ }
+ return NULL;
+}
+
+static const
+struct bcm2835_isp_fmt *find_format_by_fourcc(unsigned int fourcc,
+ struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
+ const struct bcm2835_isp_fmt *fmt;
+ unsigned int i;
+
+ for (i = 0; i < fmts->num_entries; i++) {
+ fmt = fmts->list[i];
+ if (fmt->fourcc == fourcc)
+ return fmt;
+ }
+
+ return NULL;
+}
+
+static const
+struct bcm2835_isp_fmt *find_format(struct v4l2_format *f,
+ struct bcm2835_isp_node *node)
+{
+ return find_format_by_fourcc(node_is_stats(node) ?
+ f->fmt.meta.dataformat :
+ f->fmt.pix.pixelformat,
+ node);
+}
+
+/* vb2_to_mmal_buffer() - converts vb2 buffer header to MMAL
+ *
+ * Copies all the required fields from a VB2 buffer to the MMAL buffer header,
+ * ready for sending to the VPU.
+ */
+static void vb2_to_mmal_buffer(struct mmal_buffer *buf,
+ struct vb2_v4l2_buffer *vb2)
+{
+ u64 pts;
+
+ buf->mmal_flags = 0;
+ if (vb2->flags & V4L2_BUF_FLAG_KEYFRAME)
+ buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_KEYFRAME;
+
+ /* Data must be framed correctly as one frame per buffer. */
+ buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_FRAME_END;
+
+ buf->length = vb2->vb2_buf.planes[0].bytesused;
+ /*
+ * Minor ambiguity in the V4L2 spec as to whether passing in a 0 length
+ * buffer, or one with V4L2_BUF_FLAG_LAST set denotes end of stream.
+ * Handle either.
+ */
+ if (!buf->length || vb2->flags & V4L2_BUF_FLAG_LAST)
+ buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_EOS;
+
+ /* vb2 timestamps in nsecs, mmal in usecs */
+ pts = vb2->vb2_buf.timestamp;
+ do_div(pts, 1000);
+ buf->pts = pts;
+ buf->dts = MMAL_TIME_UNKNOWN;
+}
+
+static void mmal_buffer_cb(struct vchiq_mmal_instance *instance,
+ struct vchiq_mmal_port *port, int status,
+ struct mmal_buffer *mmal_buf)
+{
+ struct bcm2835_isp_buffer *q_buf;
+ struct bcm2835_isp_node *node = port->cb_ctx;
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vb2_v4l2_buffer *vb2;
+
+ q_buf = container_of(mmal_buf, struct bcm2835_isp_buffer, mmal);
+ vb2 = &q_buf->vb;
+ v4l2_dbg(2, debug, &dev->v4l2_dev,
+ "%s: port:%s[%d], status:%d, buf:%p, dmabuf:%p, length:%lu, flags %u, pts %lld\n",
+ __func__, node_is_output(node) ? "input" : "output", node->id,
+ status, mmal_buf, mmal_buf->dma_buf, mmal_buf->length,
+ mmal_buf->mmal_flags, mmal_buf->pts);
+
+ if (mmal_buf->cmd)
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Unexpected event on output callback - %08x\n",
+ __func__, mmal_buf->cmd);
+
+ if (status) {
+ /* error in transfer */
+ if (vb2) {
+ /* there was a buffer with the error so return it */
+ vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_ERROR);
+ }
+ return;
+ }
+
+ /* vb2 timestamps in nsecs, mmal in usecs */
+ vb2->vb2_buf.timestamp = mmal_buf->pts * 1000;
+ vb2->sequence = node->sequence++;
+ vb2_set_plane_payload(&vb2->vb2_buf, 0, mmal_buf->length);
+ vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_DONE);
+
+ if (!port->enabled)
+ complete(&dev->frame_cmplt);
+}
+
+static void setup_mmal_port_format(struct bcm2835_isp_node *node,
+ struct vchiq_mmal_port *port)
+{
+ struct bcm2835_isp_q_data *q_data = &node->q_data;
+
+ port->format.encoding = q_data->fmt->mmal_fmt;
+ /* Raw image format - set width/height */
+ port->es.video.width = (q_data->bytesperline << 3) / q_data->fmt->depth;
+ port->es.video.height = q_data->height;
+ port->es.video.crop.width = q_data->width;
+ port->es.video.crop.height = q_data->height;
+ port->es.video.crop.x = 0;
+ port->es.video.crop.y = 0;
+};
+
+static int setup_mmal_port(struct bcm2835_isp_node *node)
+{
+ struct vchiq_mmal_port *port = get_port_data(node);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ unsigned int enable = 1;
+ int ret;
+
+ v4l2_dbg(2, debug, &dev->v4l2_dev, "%s: setup %s[%d]\n", __func__,
+ node->name, node->id);
+
+ vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
+ MMAL_PARAMETER_ZERO_COPY, &enable,
+ sizeof(enable));
+ setup_mmal_port_format(node, port);
+ ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
+ if (ret < 0) {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: vchiq_mmal_port_set_format failed\n",
+ __func__);
+ return ret;
+ }
+
+ if (node->q_data.sizeimage < port->minimum_buffer.size) {
+ v4l2_err(&dev->v4l2_dev,
+ "buffer size mismatch sizeimage %u < min size %u\n",
+ node->q_data.sizeimage, port->minimum_buffer.size);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
+{
+ mmal_vchi_buffer_cleanup(mmal_buf);
+
+ if (mmal_buf->dma_buf) {
+ dma_buf_put(mmal_buf->dma_buf);
+ mmal_buf->dma_buf = NULL;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_node_queue_setup(struct vb2_queue *q,
+ unsigned int *nbuffers,
+ unsigned int *nplanes,
+ unsigned int sizes[],
+ struct device *alloc_devs[])
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
+ struct vchiq_mmal_port *port;
+ unsigned int size;
+
+ if (setup_mmal_port(node))
+ return -EINVAL;
+
+ size = node->q_data.sizeimage;
+ if (size == 0) {
+ v4l2_info(&node_get_dev(node)->v4l2_dev,
+ "%s: Image size unset in queue_setup for node %s[%d]\n",
+ __func__, node->name, node->id);
+ return -EINVAL;
+ }
+
+ if (*nplanes)
+ return sizes[0] < size ? -EINVAL : 0;
+
+ *nplanes = 1;
+ sizes[0] = size;
+
+ port = get_port_data(node);
+ port->current_buffer.size = size;
+
+ if (*nbuffers < port->minimum_buffer.num)
+ *nbuffers = port->minimum_buffer.num;
+
+ port->current_buffer.num = *nbuffers;
+
+ v4l2_dbg(2, debug, &node_get_dev(node)->v4l2_dev,
+ "%s: Image size %u, nbuffers %u for node %s[%d]\n",
+ __func__, sizes[0], *nbuffers, node->name, node->id);
+ return 0;
+}
+
+static int bcm2835_isp_buf_init(struct vb2_buffer *vb)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
+ struct bcm2835_isp_buffer *buf =
+ container_of(vb2, struct bcm2835_isp_buffer, vb);
+
+ v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: vb %p\n", __func__, vb);
+
+ buf->mmal.buffer = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
+ buf->mmal.buffer_size = vb2_plane_size(&buf->vb.vb2_buf, 0);
+ mmal_vchi_buffer_init(dev->mmal_instance, &buf->mmal);
+ return 0;
+}
+
+static int bcm2835_isp_buf_prepare(struct vb2_buffer *vb)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
+ struct bcm2835_isp_buffer *buf =
+ container_of(vb2, struct bcm2835_isp_buffer, vb);
+ struct dma_buf *dma_buf;
+ int ret;
+
+ v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: type: %d ptr %p\n",
+ __func__, vb->vb2_queue->type, vb);
+
+ if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
+ if (vb2->field == V4L2_FIELD_ANY)
+ vb2->field = V4L2_FIELD_NONE;
+ if (vb2->field != V4L2_FIELD_NONE) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s field isn't supported\n", __func__);
+ return -EINVAL;
+ }
+ }
+
+ if (vb2_plane_size(vb, 0) < node->q_data.sizeimage) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s data will not fit into plane (%lu < %lu)\n",
+ __func__, vb2_plane_size(vb, 0),
+ (long)node->q_data.sizeimage);
+ return -EINVAL;
+ }
+
+ if (!V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
+ vb2_set_plane_payload(vb, 0, node->q_data.sizeimage);
+
+ switch (vb->memory) {
+ case VB2_MEMORY_DMABUF:
+ dma_buf = dma_buf_get(vb->planes[0].m.fd);
+
+ if (dma_buf != buf->mmal.dma_buf) {
+ /*
+ * dmabuf either hasn't already been mapped, or it has
+ * changed.
+ */
+ if (buf->mmal.dma_buf) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s Buffer changed - why did the core not call cleanup?\n",
+ __func__);
+ bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
+ }
+
+ buf->mmal.dma_buf = dma_buf;
+ } else {
+ /*
+ * Already have a reference to the buffer, so release it
+ * here.
+ */
+ dma_buf_put(dma_buf);
+ }
+ ret = 0;
+ break;
+ case VB2_MEMORY_MMAP:
+ /*
+ * We want to do this at init, but vb2_core_expbuf checks that
+ * the index < q->num_buffers, and q->num_buffers only gets
+ * updated once all the buffers are allocated.
+ */
+ if (!buf->mmal.dma_buf) {
+ ret = vb2_core_expbuf_dmabuf(vb->vb2_queue,
+ vb->vb2_queue->type,
+ vb->index, 0, O_CLOEXEC,
+ &buf->mmal.dma_buf);
+ v4l2_dbg(3, debug, &dev->v4l2_dev,
+ "%s: exporting ptr %p to dmabuf %p\n",
+ __func__, vb, buf->mmal.dma_buf);
+ if (ret)
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed to expbuf idx %d, ret %d\n",
+ __func__, vb->index, ret);
+ } else {
+ ret = 0;
+ }
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static void bcm2835_isp_node_buffer_queue(struct vb2_buffer *buf)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(buf->vb2_queue);
+ struct vb2_v4l2_buffer *vbuf =
+ container_of(buf, struct vb2_v4l2_buffer, vb2_buf);
+ struct bcm2835_isp_buffer *buffer =
+ container_of(vbuf, struct bcm2835_isp_buffer, vb);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: node %s[%d], buffer %p\n",
+ __func__, node->name, node->id, buffer);
+
+ vb2_to_mmal_buffer(&buffer->mmal, &buffer->vb);
+ v4l2_dbg(3, debug, &dev->v4l2_dev,
+ "%s: node %s[%d] - submitting mmal dmabuf %p\n", __func__,
+ node->name, node->id, buffer->mmal.dma_buf);
+ vchiq_mmal_submit_buffer(dev->mmal_instance, get_port_data(node),
+ &buffer->mmal);
+}
+
+static void bcm2835_isp_buffer_cleanup(struct vb2_buffer *vb)
+{
+ struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
+ struct bcm2835_isp_buffer *buffer =
+ container_of(vb2, struct bcm2835_isp_buffer, vb);
+
+ bcm2835_isp_mmal_buf_cleanup(&buffer->mmal);
+}
+
+static int bcm2835_isp_node_start_streaming(struct vb2_queue *q,
+ unsigned int count)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vchiq_mmal_port *port = get_port_data(node);
+ int ret;
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d] (count %u)\n",
+ __func__, node->name, node->id, count);
+
+ ret = vchiq_mmal_component_enable(dev->mmal_instance, dev->component);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev, "%s: Failed enabling component, ret %d\n",
+ __func__, ret);
+ return -EIO;
+ }
+
+ node->sequence = 0;
+ port->cb_ctx = node;
+ ret = vchiq_mmal_port_enable(dev->mmal_instance, port,
+ mmal_buffer_cb);
+ if (!ret)
+ atomic_inc(&dev->num_streaming);
+ else
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed enabling port, ret %d\n", __func__, ret);
+
+ return ret;
+}
+
+static void bcm2835_isp_node_stop_streaming(struct vb2_queue *q)
+{
+ struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vchiq_mmal_port *port = get_port_data(node);
+ unsigned int i;
+ int ret;
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d], mmal port %p\n",
+ __func__, node->name, node->id, port);
+
+ init_completion(&dev->frame_cmplt);
+
+ /* Disable MMAL port - this will flush buffers back */
+ ret = vchiq_mmal_port_disable(dev->mmal_instance, port);
+ if (ret)
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed disabling %s port, ret %d\n", __func__,
+ node_is_output(node) ? "i/p" : "o/p",
+ ret);
+
+ while (atomic_read(&port->buffers_with_vpu)) {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Waiting for buffers to be returned - %d outstanding\n",
+ __func__, atomic_read(&port->buffers_with_vpu));
+ ret = wait_for_completion_timeout(&dev->frame_cmplt, HZ);
+ if (ret <= 0) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Timeout waiting for buffers to be returned - %d outstanding\n",
+ __func__,
+ atomic_read(&port->buffers_with_vpu));
+ break;
+ }
+ }
+
+ /* Release the VCSM handle here to release the associated dmabuf */
+ for (i = 0; i < q->num_buffers; i++) {
+ struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(q->bufs[i]);
+ struct bcm2835_isp_buffer *buf =
+ container_of(vb2, struct bcm2835_isp_buffer, vb);
+ bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
+ }
+
+ atomic_dec(&dev->num_streaming);
+ /* If all ports disabled, then disable the component */
+ if (atomic_read(&dev->num_streaming) == 0) {
+ ret = vchiq_mmal_component_disable(dev->mmal_instance,
+ dev->component);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed disabling component, ret %d\n",
+ __func__, ret);
+ }
+ }
+
+ /*
+ * Simply wait for any vb2 buffers to finish. We could take steps to
+ * make them complete more quickly if we care, or even return them
+ * ourselves.
+ */
+ vb2_wait_for_all_buffers(&node->queue);
+}
+
+static const struct vb2_ops bcm2835_isp_node_queue_ops = {
+ .queue_setup = bcm2835_isp_node_queue_setup,
+ .buf_init = bcm2835_isp_buf_init,
+ .buf_prepare = bcm2835_isp_buf_prepare,
+ .buf_queue = bcm2835_isp_node_buffer_queue,
+ .buf_cleanup = bcm2835_isp_buffer_cleanup,
+ .start_streaming = bcm2835_isp_node_start_streaming,
+ .stop_streaming = bcm2835_isp_node_stop_streaming,
+};
+
+static const
+struct bcm2835_isp_fmt *get_default_format(struct bcm2835_isp_node *node)
+{
+ return node->supported_fmts.list[0];
+}
+
+static inline unsigned int get_bytesperline(int width,
+ const struct bcm2835_isp_fmt *fmt)
+{
+ return ALIGN((width * fmt->depth) >> 3, fmt->bytesperline_align);
+}
+
+static inline unsigned int get_sizeimage(int bpl, int width, int height,
+ const struct bcm2835_isp_fmt *fmt)
+{
+ return (bpl * height * fmt->size_multiplier_x2) >> 1;
+}
+
+static int bcm2835_isp_s_ctrl(struct v4l2_ctrl *ctrl)
+{
+ struct bcm2835_isp_dev *dev =
+ container_of(ctrl->handler, struct bcm2835_isp_dev, ctrl_handler);
+ struct bcm2835_isp_node *node = &dev->node[0];
+ int ret = 0;
+
+ /*
+ * The ISP firmware driver will ensure these settings are applied on
+ * a frame boundary, so we are safe to write them as they come in.
+ *
+ * Note that the bcm2835_isp_* param structures are identical to the
+ * mmal-parameters.h definitions. This avoids the need for unnecessary
+ * field-by-field copying between structures.
+ */
+ switch (ctrl->id) {
+ case V4L2_CID_RED_BALANCE:
+ dev->r_gain = ctrl->val;
+ ret = set_wb_gains(node);
+ break;
+ case V4L2_CID_BLUE_BALANCE:
+ dev->b_gain = ctrl->val;
+ ret = set_wb_gains(node);
+ break;
+ case V4L2_CID_DIGITAL_GAIN:
+ ret = set_digital_gain(node, ctrl->val);
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_CC_MATRIX:
+ ret = set_isp_param(node, MMAL_PARAMETER_CUSTOM_CCM,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_custom_ccm));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_LENS_SHADING:
+ ret = set_isp_param(node, MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_lens_shading));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL:
+ ret = set_isp_param(node, MMAL_PARAMETER_BLACK_LEVEL,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_black_level));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_GEQ:
+ ret = set_isp_param(node, MMAL_PARAMETER_GEQ,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_geq));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_GAMMA:
+ ret = set_isp_param(node, MMAL_PARAMETER_GAMMA,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_gamma));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_DENOISE:
+ ret = set_isp_param(node, MMAL_PARAMETER_DENOISE,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_denoise));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_SHARPEN:
+ ret = set_isp_param(node, MMAL_PARAMETER_SHARPEN,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_sharpen));
+ break;
+ case V4L2_CID_USER_BCM2835_ISP_DPC:
+ ret = set_isp_param(node, MMAL_PARAMETER_DPC,
+ ctrl->p_new.p_u8,
+ sizeof(struct bcm2835_isp_dpc));
+ break;
+ default:
+ v4l2_info(&dev->v4l2_dev, "Unrecognised control\n");
+ ret = -EINVAL;
+ }
+
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev, "%s: Failed setting ctrl \"%s\" (%08x), err %d\n",
+ __func__, ctrl->name, ctrl->id, ret);
+ ret = -EIO;
+ }
+
+ return ret;
+}
+
+static const struct v4l2_ctrl_ops bcm2835_isp_ctrl_ops = {
+ .s_ctrl = bcm2835_isp_s_ctrl,
+};
+
+static const struct v4l2_file_operations bcm2835_isp_fops = {
+ .owner = THIS_MODULE,
+ .open = v4l2_fh_open,
+ .release = vb2_fop_release,
+ .poll = vb2_fop_poll,
+ .unlocked_ioctl = video_ioctl2,
+ .mmap = vb2_fop_mmap
+};
+
+static int populate_qdata_fmt(struct v4l2_format *f,
+ struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct bcm2835_isp_q_data *q_data = &node->q_data;
+ struct vchiq_mmal_port *port;
+ int ret;
+
+ if (!node_is_stats(node)) {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Setting pix format for type %d, wxh: %ux%u, fmt: %08x, size %u\n",
+ __func__, f->type, f->fmt.pix.width, f->fmt.pix.height,
+ f->fmt.pix.pixelformat, f->fmt.pix.sizeimage);
+
+ q_data->fmt = find_format(f, node);
+ q_data->width = f->fmt.pix.width;
+ q_data->height = f->fmt.pix.height;
+ q_data->height = f->fmt.pix.height;
+
+ /* All parameters should have been set correctly by try_fmt */
+ q_data->bytesperline = f->fmt.pix.bytesperline;
+ q_data->sizeimage = f->fmt.pix.sizeimage;
+ } else {
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Setting meta format for fmt: %08x, size %u\n",
+ __func__, f->fmt.meta.dataformat,
+ f->fmt.meta.buffersize);
+
+ q_data->fmt = find_format(f, node);
+ q_data->width = 0;
+ q_data->height = 0;
+ q_data->bytesperline = 0;
+ q_data->sizeimage = f->fmt.meta.buffersize;
+ }
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Calculated bpl as %u, size %u\n", __func__,
+ q_data->bytesperline, q_data->sizeimage);
+
+ /* If we have a component then setup the port as well */
+ port = get_port_data(node);
+ setup_mmal_port_format(node, port);
+ ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Failed vchiq_mmal_port_set_format on port, ret %d\n",
+ __func__, ret);
+ ret = -EINVAL;
+ }
+
+ if (q_data->sizeimage < port->minimum_buffer.size) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: Current buffer size of %u < min buf size %u - driver mismatch to MMAL\n",
+ __func__,
+ q_data->sizeimage,
+ port->minimum_buffer.size);
+ }
+
+ v4l2_dbg(1, debug, &dev->v4l2_dev,
+ "%s: Set format for type %d, wxh: %dx%d, fmt: %08x, size %u\n",
+ __func__, f->type, q_data->width, q_data->height,
+ q_data->fmt->fourcc, q_data->sizeimage);
+
+ return ret;
+}
+
+static int bcm2835_isp_node_querycap(struct file *file, void *priv,
+ struct v4l2_capability *cap)
+{
+ strscpy(cap->driver, BCM2835_ISP_NAME, sizeof(cap->driver));
+ strscpy(cap->card, BCM2835_ISP_NAME, sizeof(cap->card));
+ snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
+ BCM2835_ISP_NAME);
+
+ return 0;
+}
+
+static int bcm2835_isp_node_g_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ if (node_is_stats(node)) {
+ f->fmt.meta.dataformat = V4L2_META_FMT_BCM2835_ISP_STATS;
+ f->fmt.meta.buffersize =
+ get_port_data(node)->minimum_buffer.size;
+ } else {
+ struct bcm2835_isp_q_data *q_data = &node->q_data;
+
+ f->fmt.pix.width = q_data->width;
+ f->fmt.pix.height = q_data->height;
+ f->fmt.pix.field = V4L2_FIELD_NONE;
+ f->fmt.pix.pixelformat = q_data->fmt->fourcc;
+ f->fmt.pix.bytesperline = q_data->bytesperline;
+ f->fmt.pix.sizeimage = q_data->sizeimage;
+ f->fmt.pix.colorspace = q_data->fmt->colorspace;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_node_enum_fmt(struct file *file, void *priv,
+ struct v4l2_fmtdesc *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ if (f->index < fmts->num_entries) {
+ /* Format found */
+ f->pixelformat = fmts->list[f->index]->fourcc;
+ f->flags = fmts->list[f->index]->flags;
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static int bcm2835_isp_enum_framesizes(struct file *file, void *priv,
+ struct v4l2_frmsizeenum *fsize)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ const struct bcm2835_isp_fmt *fmt;
+
+ if (node_is_stats(node) || fsize->index)
+ return -EINVAL;
+
+ fmt = find_format_by_fourcc(fsize->pixel_format, node);
+ if (!fmt) {
+ v4l2_err(&dev->v4l2_dev, "Invalid pixel code: %x\n",
+ fsize->pixel_format);
+ return -EINVAL;
+ }
+
+ fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
+ fsize->stepwise.min_width = MIN_DIM;
+ fsize->stepwise.max_width = MAX_DIM;
+ fsize->stepwise.step_width = fmt->step_size;
+
+ fsize->stepwise.min_height = MIN_DIM;
+ fsize->stepwise.max_height = MAX_DIM;
+ fsize->stepwise.step_height = fmt->step_size;
+
+ return 0;
+}
+
+static int bcm2835_isp_node_try_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ const struct bcm2835_isp_fmt *fmt;
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ fmt = find_format(f, node);
+ if (!fmt)
+ fmt = get_default_format(node);
+
+ if (!node_is_stats(node)) {
+ f->fmt.pix.width = max(min(f->fmt.pix.width, MAX_DIM),
+ MIN_DIM);
+ f->fmt.pix.height = max(min(f->fmt.pix.height, MAX_DIM),
+ MIN_DIM);
+
+ f->fmt.pix.pixelformat = fmt->fourcc;
+ f->fmt.pix.colorspace = fmt->colorspace;
+ f->fmt.pix.bytesperline = get_bytesperline(f->fmt.pix.width,
+ fmt);
+ f->fmt.pix.field = V4L2_FIELD_NONE;
+ f->fmt.pix.sizeimage =
+ get_sizeimage(f->fmt.pix.bytesperline, f->fmt.pix.width,
+ f->fmt.pix.height, fmt);
+ } else {
+ f->fmt.meta.dataformat = fmt->fourcc;
+ f->fmt.meta.buffersize =
+ get_port_data(node)->minimum_buffer.size;
+ }
+
+ return 0;
+}
+
+static int bcm2835_isp_node_s_fmt(struct file *file, void *priv,
+ struct v4l2_format *f)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ int ret;
+
+ if (f->type != node->queue.type)
+ return -EINVAL;
+
+ ret = bcm2835_isp_node_try_fmt(file, priv, f);
+ if (ret)
+ return ret;
+
+ v4l2_dbg(1, debug, &node_get_dev(node)->v4l2_dev,
+ "%s: Set format for node %s[%d]\n",
+ __func__, node->name, node->id);
+
+ return populate_qdata_fmt(f, node);
+}
+
+static int bcm2835_isp_node_s_selection(struct file *file, void *fh,
+ struct v4l2_selection *s)
+{
+ struct mmal_parameter_crop crop;
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vchiq_mmal_port *port = get_port_data(node);
+
+ /* This return value is required fro V4L2 compliance. */
+ if (node_is_stats(node))
+ return -ENOTTY;
+
+ if (!s->r.width || !s->r.height)
+ return -EINVAL;
+
+ /* Adjust the crop window if goes outside the frame dimensions. */
+ s->r.left = min((unsigned int)max(s->r.left, 0),
+ node->q_data.width - MIN_DIM);
+ s->r.top = min((unsigned int)max(s->r.top, 0),
+ node->q_data.height - MIN_DIM);
+ s->r.width = max(min(s->r.width, node->q_data.width - s->r.left),
+ MIN_DIM);
+ s->r.height = max(min(s->r.height, node->q_data.height - s->r.top),
+ MIN_DIM);
+
+ crop.rect.x = s->r.left;
+ crop.rect.y = s->r.top;
+ crop.rect.width = s->r.width;
+ crop.rect.height = s->r.height;
+
+ return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
+ MMAL_PARAMETER_CROP,
+ &crop, sizeof(crop));
+}
+
+static int bcm2835_isp_node_g_selection(struct file *file, void *fh,
+ struct v4l2_selection *s)
+{
+ struct bcm2835_isp_node *node = video_drvdata(file);
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct vchiq_mmal_port *port = get_port_data(node);
+ struct mmal_parameter_crop crop;
+ u32 crop_size = sizeof(crop);
+ int ret;
+
+ /* This return value is required for V4L2 compliance. */
+ if (node_is_stats(node))
+ return -ENOTTY;
+
+ /* We can only return out an input crop. */
+ if (s->target != V4L2_SEL_TGT_CROP)
+ return -EINVAL;
+
+ ret = vchiq_mmal_port_parameter_get(dev->mmal_instance, port,
+ MMAL_PARAMETER_CROP,
+ &crop, &crop_size);
+ if (!ret)
+ return -EINVAL;
+
+ s->r.left = crop.rect.x;
+ s->r.top = crop.rect.y;
+ s->r.width = crop.rect.width;
+ s->r.height = crop.rect.height;
+
+ return 0;
+}
+
+static int bcm3285_isp_subscribe_event(struct v4l2_fh *fh,
+ const struct v4l2_event_subscription *s)
+{
+ switch (s->type) {
+ /* Cannot change source parameters dynamically at runtime. */
+ case V4L2_EVENT_SOURCE_CHANGE:
+ return -EINVAL;
+ case V4L2_EVENT_CTRL:
+ return v4l2_ctrl_subscribe_event(fh, s);
+ default:
+ return v4l2_event_subscribe(fh, s, 4, NULL);
+ }
+}
+
+static const struct v4l2_ioctl_ops bcm2835_isp_node_ioctl_ops = {
+ .vidioc_querycap = bcm2835_isp_node_querycap,
+ .vidioc_g_fmt_vid_cap = bcm2835_isp_node_g_fmt,
+ .vidioc_g_fmt_vid_out = bcm2835_isp_node_g_fmt,
+ .vidioc_g_fmt_meta_cap = bcm2835_isp_node_g_fmt,
+ .vidioc_s_fmt_vid_cap = bcm2835_isp_node_s_fmt,
+ .vidioc_s_fmt_vid_out = bcm2835_isp_node_s_fmt,
+ .vidioc_s_fmt_meta_cap = bcm2835_isp_node_s_fmt,
+ .vidioc_try_fmt_vid_cap = bcm2835_isp_node_try_fmt,
+ .vidioc_try_fmt_vid_out = bcm2835_isp_node_try_fmt,
+ .vidioc_try_fmt_meta_cap = bcm2835_isp_node_try_fmt,
+ .vidioc_s_selection = bcm2835_isp_node_s_selection,
+ .vidioc_g_selection = bcm2835_isp_node_g_selection,
+
+ .vidioc_enum_fmt_vid_cap = bcm2835_isp_node_enum_fmt,
+ .vidioc_enum_fmt_vid_out = bcm2835_isp_node_enum_fmt,
+ .vidioc_enum_fmt_meta_cap = bcm2835_isp_node_enum_fmt,
+ .vidioc_enum_framesizes = bcm2835_isp_enum_framesizes,
+
+ .vidioc_reqbufs = vb2_ioctl_reqbufs,
+ .vidioc_querybuf = vb2_ioctl_querybuf,
+ .vidioc_qbuf = vb2_ioctl_qbuf,
+ .vidioc_dqbuf = vb2_ioctl_dqbuf,
+ .vidioc_expbuf = vb2_ioctl_expbuf,
+ .vidioc_create_bufs = vb2_ioctl_create_bufs,
+ .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
+
+ .vidioc_streamon = vb2_ioctl_streamon,
+ .vidioc_streamoff = vb2_ioctl_streamoff,
+
+ .vidioc_subscribe_event = bcm3285_isp_subscribe_event,
+ .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+};
+
+/*
+ * Size of the array to provide to the VPU when asking for the list of supported
+ * formats.
+ *
+ * The ISP component currently advertises 33 input formats, so add a small
+ * overhead on that.
+ */
+#define MAX_SUPPORTED_ENCODINGS 40
+
+/* Populate node->supported_fmts with the formats supported by those ports. */
+static int bcm2835_isp_get_supported_fmts(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+ struct bcm2835_isp_fmt const **list;
+ unsigned int i, j, num_encodings;
+ u32 fourccs[MAX_SUPPORTED_ENCODINGS];
+ u32 param_size = sizeof(fourccs);
+ int ret;
+
+ ret = vchiq_mmal_port_parameter_get(dev->mmal_instance,
+ get_port_data(node),
+ MMAL_PARAMETER_SUPPORTED_ENCODINGS,
+ &fourccs, ¶m_size);
+
+ if (ret) {
+ if (ret == MMAL_MSG_STATUS_ENOSPC) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: port has more encoding than we provided space for. Some are dropped.\n",
+ __func__);
+ num_encodings = MAX_SUPPORTED_ENCODINGS;
+ } else {
+ v4l2_err(&dev->v4l2_dev, "%s: get_param ret %u.\n",
+ __func__, ret);
+ return -EINVAL;
+ }
+ } else {
+ num_encodings = param_size / sizeof(u32);
+ }
+
+ /*
+ * Assume at this stage that all encodings will be supported in V4L2.
+ * Any that aren't supported will waste a very small amount of memory.
+ */
+ list = devm_kzalloc(dev->dev,
+ sizeof(struct bcm2835_isp_fmt *) * num_encodings,
+ GFP_KERNEL);
+ if (!list)
+ return -ENOMEM;
+ node->supported_fmts.list = list;
+
+ for (i = 0, j = 0; i < num_encodings; i++) {
+ const struct bcm2835_isp_fmt *fmt = get_fmt(fourccs[i]);
+
+ if (fmt) {
+ list[j] = fmt;
+ j++;
+ }
+ }
+ node->supported_fmts.num_entries = j;
+
+ return 0;
+}
+
+/*
+ * Register a device node /dev/video<N> to go along with one of the ISP's input
+ * or output nodes.
+ */
+static int register_node(struct bcm2835_isp_dev *dev,
+ struct bcm2835_isp_node *node,
+ int index)
+{
+ struct video_device *vfd;
+ struct vb2_queue *queue;
+ int ret;
+
+ mutex_init(&node->lock);
+
+ node->dev = dev;
+ vfd = &node->vfd;
+ queue = &node->queue;
+ queue->type = index_to_queue_type(index);
+ /*
+ * Setup the node type-specific params.
+ *
+ * Only the OUTPUT node can set controls and crop windows. However,
+ * we must allow the s/g_selection ioctl on the stats node as v4l2
+ * compliance expects it to return a -ENOTTY, and the framework
+ * does not handle it if the ioctl is disabled.
+ */
+ switch (queue->type) {
+ case V4L2_BUF_TYPE_VIDEO_OUTPUT:
+ vfd->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
+ node->id = index;
+ node->vfl_dir = VFL_DIR_TX;
+ node->name = "output";
+ break;
+ case V4L2_BUF_TYPE_VIDEO_CAPTURE:
+ vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
+ /* First Capture node starts at id 0, etc. */
+ node->id = index - BCM2835_ISP_NUM_OUTPUTS;
+ node->vfl_dir = VFL_DIR_RX;
+ node->name = "capture";
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_S_SELECTION);
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_G_SELECTION);
+ break;
+ case V4L2_BUF_TYPE_META_CAPTURE:
+ vfd->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
+ node->id = index - BCM2835_ISP_NUM_OUTPUTS;
+ node->vfl_dir = VFL_DIR_RX;
+ node->name = "stats";
+ v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
+ break;
+ }
+
+ /* We use the selection API instead of the old crop API. */
+ v4l2_disable_ioctl(vfd, VIDIOC_CROPCAP);
+ v4l2_disable_ioctl(vfd, VIDIOC_G_CROP);
+ v4l2_disable_ioctl(vfd, VIDIOC_S_CROP);
+
+ ret = bcm2835_isp_get_supported_fmts(node);
+ if (ret)
+ return ret;
+
+ /* Initialise the the video node. */
+ vfd->vfl_type = VFL_TYPE_VIDEO;
+ vfd->fops = &bcm2835_isp_fops,
+ vfd->ioctl_ops = &bcm2835_isp_node_ioctl_ops,
+ vfd->minor = -1,
+ vfd->release = video_device_release_empty,
+ vfd->queue = &node->queue;
+ vfd->lock = &node->lock;
+ vfd->v4l2_dev = &dev->v4l2_dev;
+ vfd->vfl_dir = node->vfl_dir;
+
+ node->q_data.fmt = get_default_format(node);
+ node->q_data.width = DEFAULT_DIM;
+ node->q_data.height = DEFAULT_DIM;
+ node->q_data.bytesperline =
+ get_bytesperline(DEFAULT_DIM, node->q_data.fmt);
+ node->q_data.sizeimage = node_is_stats(node) ?
+ get_port_data(node)->recommended_buffer.size :
+ get_sizeimage(node->q_data.bytesperline,
+ node->q_data.width,
+ node->q_data.height,
+ node->q_data.fmt);
+
+ queue->io_modes = VB2_MMAP | VB2_DMABUF;
+ queue->drv_priv = node;
+ queue->ops = &bcm2835_isp_node_queue_ops;
+ queue->mem_ops = &vb2_dma_contig_memops;
+ queue->buf_struct_size = sizeof(struct bcm2835_isp_buffer);
+ queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
+ queue->dev = dev->dev;
+ queue->lock = &node->queue_lock;
+
+ ret = vb2_queue_init(queue);
+ if (ret < 0) {
+ v4l2_info(&dev->v4l2_dev, "vb2_queue_init failed\n");
+ return ret;
+ }
+ node->queue_init = true;
+
+ /* Define the device names */
+ snprintf(vfd->name, sizeof(node->vfd.name), "%s-%s%d", BCM2835_ISP_NAME,
+ node->name, node->id);
+
+ ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr + index);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "Failed to register video %s[%d] device node\n",
+ node->name, node->id);
+ return ret;
+ }
+
+ node->registered = true;
+ video_set_drvdata(vfd, node);
+
+ /* Set some controls and defaults, but only on the VIDEO_OUTPUT node. */
+ if (node_is_output(node)) {
+ unsigned int i;
+
+ /* Use this ctrl template to assign all out ISP custom ctrls. */
+ struct v4l2_ctrl_config ctrl_template = {
+ .ops = &bcm2835_isp_ctrl_ops,
+ .type = V4L2_CTRL_TYPE_U8,
+ .def = 0,
+ .min = 0x00,
+ .max = 0xff,
+ .step = 1,
+ };
+
+ v4l2_ctrl_handler_init(&dev->ctrl_handler, 4);
+
+ dev->r_gain = 1000;
+ dev->b_gain = 1000;
+
+ v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
+ V4L2_CID_RED_BALANCE, 1, 0xffff, 1,
+ dev->r_gain);
+
+ v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
+ V4L2_CID_BLUE_BALANCE, 1, 0xffff, 1,
+ dev->b_gain);
+
+ v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
+ V4L2_CID_DIGITAL_GAIN, 1, 0xffff, 1, 1000);
+
+ for (i = 0; i < ARRAY_SIZE(custom_ctrls); i++) {
+ ctrl_template.name = custom_ctrls[i].name;
+ ctrl_template.id = custom_ctrls[i].id;
+ ctrl_template.dims[0] = custom_ctrls[i].size;
+ ctrl_template.flags = custom_ctrls[i].flags;
+ v4l2_ctrl_new_custom(&dev->ctrl_handler,
+ &ctrl_template, NULL);
+ }
+
+ node->vfd.ctrl_handler = &dev->ctrl_handler;
+ }
+
+ v4l2_info(&dev->v4l2_dev,
+ "Device node %s[%d] registered as /dev/video%d\n",
+ node->name, node->id, vfd->num);
+
+ return 0;
+}
+
+/* Unregister one of the /dev/video<N> nodes associated with the ISP. */
+static void unregister_node(struct bcm2835_isp_node *node)
+{
+ struct bcm2835_isp_dev *dev = node_get_dev(node);
+
+ v4l2_info(&dev->v4l2_dev,
+ "Unregistering node %s[%d] device node /dev/video%d\n",
+ node->name, node->id, node->vfd.num);
+
+ if (node->queue_init)
+ vb2_queue_release(&node->queue);
+
+ if (node->registered) {
+ video_unregister_device(&node->vfd);
+ if (node_is_output(node))
+ v4l2_ctrl_handler_free(&dev->ctrl_handler);
+ }
+
+ /*
+ * node->supported_fmts.list is free'd automatically
+ * as a managed resource.
+ */
+ node->supported_fmts.list = NULL;
+ node->supported_fmts.num_entries = 0;
+ node->vfd.ctrl_handler = NULL;
+ node->registered = false;
+ node->queue_init = false;
+}
+
+static void media_controller_unregister(struct bcm2835_isp_dev *dev)
+{
+ unsigned int i;
+
+ v4l2_info(&dev->v4l2_dev, "Unregister from media controller\n");
+
+ if (dev->media_device_registered) {
+ media_device_unregister(&dev->mdev);
+ media_device_cleanup(&dev->mdev);
+ dev->media_device_registered = false;
+ }
+
+ kfree(dev->entity.name);
+ dev->entity.name = NULL;
+
+ if (dev->media_entity_registered) {
+ media_device_unregister_entity(&dev->entity);
+ dev->media_entity_registered = false;
+ }
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ struct bcm2835_isp_node *node = &dev->node[i];
+
+ if (node->media_node_registered) {
+ media_remove_intf_links(node->intf_link->intf);
+ media_entity_remove_links(&dev->node[i].vfd.entity);
+ media_devnode_remove(node->intf_devnode);
+ media_device_unregister_entity(&node->vfd.entity);
+ kfree(node->vfd.entity.name);
+ }
+ node->media_node_registered = false;
+ }
+
+ dev->v4l2_dev.mdev = NULL;
+}
+
+static int media_controller_register_node(struct bcm2835_isp_dev *dev, int num)
+{
+ struct bcm2835_isp_node *node = &dev->node[num];
+ struct media_entity *entity = &node->vfd.entity;
+ int output = node_is_output(node);
+ char *name;
+ int ret;
+
+ v4l2_info(&dev->v4l2_dev,
+ "Register %s node %d with media controller\n",
+ output ? "output" : "capture", num);
+ entity->obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
+ entity->function = MEDIA_ENT_F_IO_V4L;
+ entity->info.dev.major = VIDEO_MAJOR;
+ entity->info.dev.minor = node->vfd.minor;
+ name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
+ if (!name) {
+ ret = -ENOMEM;
+ goto error_no_mem;
+ }
+ snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "%s0-%s%d",
+ BCM2835_ISP_NAME, output ? "output" : "capture", num);
+ entity->name = name;
+ node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
+ ret = media_entity_pads_init(entity, 1, &node->pad);
+ if (ret)
+ goto error_pads_init;
+ ret = media_device_register_entity(&dev->mdev, entity);
+ if (ret)
+ goto error_register_entity;
+
+ node->intf_devnode = media_devnode_create(&dev->mdev,
+ MEDIA_INTF_T_V4L_VIDEO, 0,
+ VIDEO_MAJOR, node->vfd.minor);
+ if (!node->intf_devnode) {
+ ret = -ENOMEM;
+ goto error_devnode_create;
+ }
+
+ node->intf_link = media_create_intf_link(entity,
+ &node->intf_devnode->intf,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (!node->intf_link) {
+ ret = -ENOMEM;
+ goto error_create_intf_link;
+ }
+
+ if (output)
+ ret = media_create_pad_link(entity, 0, &dev->entity, num,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ else
+ ret = media_create_pad_link(&dev->entity, num, entity, 0,
+ MEDIA_LNK_FL_IMMUTABLE |
+ MEDIA_LNK_FL_ENABLED);
+ if (ret)
+ goto error_create_pad_link;
+
+ dev->node[num].media_node_registered = true;
+ return 0;
+
+error_create_pad_link:
+ media_remove_intf_links(&node->intf_devnode->intf);
+error_create_intf_link:
+ media_devnode_remove(node->intf_devnode);
+error_devnode_create:
+ media_device_unregister_entity(&node->vfd.entity);
+error_register_entity:
+error_pads_init:
+ kfree(entity->name);
+ entity->name = NULL;
+error_no_mem:
+ if (ret)
+ v4l2_info(&dev->v4l2_dev, "Error registering node\n");
+
+ return ret;
+}
+
+static int media_controller_register(struct bcm2835_isp_dev *dev)
+{
+ char *name;
+ unsigned int i;
+ int ret;
+
+ v4l2_dbg(2, debug, &dev->v4l2_dev, "Registering with media controller\n");
+ dev->mdev.dev = dev->dev;
+ strscpy(dev->mdev.model, "bcm2835-isp",
+ sizeof(dev->mdev.model));
+ strscpy(dev->mdev.bus_info, "platform:bcm2835-isp",
+ sizeof(dev->mdev.bus_info));
+ media_device_init(&dev->mdev);
+ dev->v4l2_dev.mdev = &dev->mdev;
+
+ v4l2_dbg(2, debug, &dev->v4l2_dev, "Register entity for nodes\n");
+
+ name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
+ if (!name) {
+ ret = -ENOMEM;
+ goto done;
+ }
+ snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "bcm2835_isp0");
+ dev->entity.name = name;
+ dev->entity.obj_type = MEDIA_ENTITY_TYPE_BASE;
+ dev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ dev->pad[i].flags = node_is_output(&dev->node[i]) ?
+ MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
+ }
+
+ ret = media_entity_pads_init(&dev->entity, BCM2835_ISP_NUM_NODES,
+ dev->pad);
+ if (ret)
+ goto done;
+
+ ret = media_device_register_entity(&dev->mdev, &dev->entity);
+ if (ret)
+ goto done;
+
+ dev->media_entity_registered = true;
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ ret = media_controller_register_node(dev, i);
+ if (ret)
+ goto done;
+ }
+
+ ret = media_device_register(&dev->mdev);
+ if (!ret)
+ dev->media_device_registered = true;
+done:
+ return ret;
+}
+
+static int bcm2835_isp_remove(struct platform_device *pdev)
+{
+ struct bcm2835_isp_dev *dev = platform_get_drvdata(pdev);
+ unsigned int i;
+
+ media_controller_unregister(dev);
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++)
+ unregister_node(&dev->node[i]);
+
+ v4l2_device_unregister(&dev->v4l2_dev);
+
+ if (dev->component)
+ vchiq_mmal_component_finalise(dev->mmal_instance,
+ dev->component);
+
+ vchiq_mmal_finalise(dev->mmal_instance);
+
+ return 0;
+}
+
+static int bcm2835_isp_probe(struct platform_device *pdev)
+{
+ struct bcm2835_isp_dev *dev;
+ unsigned int i;
+ int ret;
+
+ dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ dev->dev = &pdev->dev;
+
+ ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
+ if (ret)
+ return ret;
+
+ ret = vchiq_mmal_init(&dev->mmal_instance);
+ if (ret) {
+ v4l2_device_unregister(&dev->v4l2_dev);
+ return ret;
+ }
+
+ ret = vchiq_mmal_component_init(dev->mmal_instance, "ril.isp",
+ &dev->component);
+ if (ret) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: failed to create ril.isp component\n", __func__);
+ goto error;
+ }
+
+ if ((dev->component->inputs != BCM2835_ISP_NUM_OUTPUTS) ||
+ (dev->component->outputs != BCM2835_ISP_NUM_CAPTURES +
+ BCM2835_ISP_NUM_METADATA)) {
+ v4l2_err(&dev->v4l2_dev,
+ "%s: ril.isp returned %d i/p (%d expected), %d o/p (%d expected) ports\n",
+ __func__, dev->component->inputs,
+ BCM2835_ISP_NUM_OUTPUTS,
+ dev->component->outputs,
+ BCM2835_ISP_NUM_CAPTURES + BCM2835_ISP_NUM_METADATA);
+ goto error;
+ }
+
+ atomic_set(&dev->num_streaming, 0);
+
+ for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
+ struct bcm2835_isp_node *node = &dev->node[i];
+
+ ret = register_node(dev, node, i);
+ if (ret)
+ goto error;
+ }
+
+ ret = media_controller_register(dev);
+ if (ret)
+ goto error;
+
+ platform_set_drvdata(pdev, dev);
+ v4l2_info(&dev->v4l2_dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
+ return 0;
+
+error:
+ bcm2835_isp_remove(pdev);
+
+ return ret;
+}
+
+static struct platform_driver bcm2835_isp_pdrv = {
+ .probe = bcm2835_isp_probe,
+ .remove = bcm2835_isp_remove,
+ .driver = {
+ .name = BCM2835_ISP_NAME,
+ },
+};
+
+module_platform_driver(bcm2835_isp_pdrv);
+
+MODULE_DESCRIPTION("BCM2835 ISP driver");
+MODULE_AUTHOR("Naushir Patuck <naush@raspberrypi.com>");
+MODULE_LICENSE("GPL");
+MODULE_VERSION("1.0");
+MODULE_ALIAS("platform:bcm2835-isp");
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
new file mode 100644
index 000000000000..cfbb1063aad1
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Broadcom BCM2835 ISP driver
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#ifndef BCM2835_ISP_CTRLS
+#define BCM2835_ISP_CTRLS
+
+#include <linux/bcm2835-isp.h>
+
+struct bcm2835_isp_custom_ctrl {
+ const char *name;
+ u32 id;
+ u32 size;
+ u32 flags;
+};
+
+static const struct bcm2835_isp_custom_ctrl custom_ctrls[] = {
+ {
+ .name = "Colour Correction Matrix",
+ .id = V4L2_CID_USER_BCM2835_ISP_CC_MATRIX,
+ .size = sizeof(struct bcm2835_isp_custom_ccm),
+ .flags = 0
+ }, {
+ .name = "Lens Shading",
+ .id = V4L2_CID_USER_BCM2835_ISP_LENS_SHADING,
+ .size = sizeof(struct bcm2835_isp_lens_shading),
+ .flags = V4L2_CTRL_FLAG_EXECUTE_ON_WRITE
+ }, {
+ .name = "Black Level",
+ .id = V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL,
+ .size = sizeof(struct bcm2835_isp_black_level),
+ .flags = 0
+ }, {
+ .name = "Green Equalisation",
+ .id = V4L2_CID_USER_BCM2835_ISP_GEQ,
+ .size = sizeof(struct bcm2835_isp_geq),
+ .flags = 0
+ }, {
+ .name = "Gamma",
+ .id = V4L2_CID_USER_BCM2835_ISP_GAMMA,
+ .size = sizeof(struct bcm2835_isp_gamma),
+ .flags = 0
+ }, {
+ .name = "Sharpen",
+ .id = V4L2_CID_USER_BCM2835_ISP_SHARPEN,
+ .size = sizeof(struct bcm2835_isp_sharpen),
+ .flags = 0
+ }, {
+ .name = "Denoise",
+ .id = V4L2_CID_USER_BCM2835_ISP_DENOISE,
+ .size = sizeof(struct bcm2835_isp_denoise),
+ .flags = 0
+ }, {
+ .name = "Defective Pixel Correction",
+ .id = V4L2_CID_USER_BCM2835_ISP_DPC,
+ .size = sizeof(struct bcm2835_isp_dpc),
+ .flags = 0
+ }
+};
+
+#endif
diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
new file mode 100644
index 000000000000..af3bde152bb2
--- /dev/null
+++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
@@ -0,0 +1,301 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Broadcom BCM2835 ISP driver
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#ifndef BCM2835_ISP_FMTS
+#define BCM2835_ISP_FMTS
+
+#include <linux/videodev2.h>
+#include "vchiq-mmal/mmal-encodings.h"
+
+struct bcm2835_isp_fmt {
+ u32 fourcc;
+ int depth;
+ int bytesperline_align;
+ u32 flags;
+ u32 mmal_fmt;
+ int size_multiplier_x2;
+ enum v4l2_colorspace colorspace;
+ unsigned int step_size;
+};
+
+struct bcm2835_isp_fmt_list {
+ struct bcm2835_isp_fmt const **list;
+ unsigned int num_entries;
+};
+
+static const struct bcm2835_isp_fmt supported_formats[] = {
+ {
+ /* YUV formats */
+ .fourcc = V4L2_PIX_FMT_YUV420,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_I420,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YVU420,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_YV12,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV12,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_NV12,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_NV21,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_NV21,
+ .size_multiplier_x2 = 3,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YUYV,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_YUYV,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_UYVY,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_UYVY,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_YVYU,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_YVYU,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_VYUY,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_VYUY,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SMPTE170M,
+ .step_size = 2,
+ }, {
+ /* RGB formats */
+ .fourcc = V4L2_PIX_FMT_RGB24,
+ .depth = 24,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_RGB24,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_RGB565,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_RGB16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_BGR24,
+ .depth = 24,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BGR24,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ .fourcc = V4L2_PIX_FMT_ABGR32,
+ .depth = 32,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BGRA,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_SRGB,
+ .step_size = 1,
+ }, {
+ /* Bayer formats */
+ /* 8 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG8,
+ .depth = 8,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG8,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 10 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG10P,
+ .depth = 10,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 12 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG12P,
+ .depth = 12,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12P,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* 16 bit */
+ .fourcc = V4L2_PIX_FMT_SRGGB16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SBGGR16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGRBG16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ .fourcc = V4L2_PIX_FMT_SGBRG16,
+ .depth = 16,
+ .bytesperline_align = 32,
+ .flags = 0,
+ .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG16,
+ .size_multiplier_x2 = 2,
+ .colorspace = V4L2_COLORSPACE_RAW,
+ .step_size = 2,
+ }, {
+ /* ISP statistics format */
+ .fourcc = V4L2_META_FMT_BCM2835_ISP_STATS,
+ .mmal_fmt = MMAL_ENCODING_BRCM_STATS,
+ /* The rest are not valid fields for stats. */
+ }
+};
+
+#endif
diff --git a/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
new file mode 100644
index 000000000000..edc452fa8318
--- /dev/null
+++ b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
@@ -0,0 +1,333 @@
+/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
+/*
+ * bcm2835-isp.h
+ *
+ * BCM2835 ISP driver - user space header file.
+ *
+ * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
+ *
+ * Author: Naushir Patuck (naush@raspberrypi.com)
+ *
+ */
+
+#ifndef __BCM2835_ISP_H_
+#define __BCM2835_ISP_H_
+
+#include <linux/v4l2-controls.h>
+
+/* TODO: move the control IDs definitions to v4l2-controls.h */
+#define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10c0)
+
+/* TODO: move the formats definitions to videodev2.h */
+/* 12 Y/CbCr 4:2:0 128 pixel wide column */
+#define V4L2_PIX_FMT_NV12_COL128 v4l2_fourcc('N', 'C', '1', '2')
+/* Y/CbCr 4:2:0 10bpc, 3x10 packed as 4 bytes in a 128 bytes / 96 pixel wide column */
+#define V4L2_PIX_FMT_NV12_10_COL128 v4l2_fourcc('N', 'C', '3', '0')
+/* Sensor Ancillary metadata */
+#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S')
+/* BCM2835 ISP image statistics output */
+#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A')
+
+#define V4L2_CID_USER_BCM2835_ISP_CC_MATRIX \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0001)
+#define V4L2_CID_USER_BCM2835_ISP_LENS_SHADING \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0002)
+#define V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0003)
+#define V4L2_CID_USER_BCM2835_ISP_GEQ \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0004)
+#define V4L2_CID_USER_BCM2835_ISP_GAMMA \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0005)
+#define V4L2_CID_USER_BCM2835_ISP_DENOISE \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0006)
+#define V4L2_CID_USER_BCM2835_ISP_SHARPEN \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
+#define V4L2_CID_USER_BCM2835_ISP_DPC \
+ (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
+
+/*
+ * All structs below are directly mapped onto the equivalent structs in
+ * drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
+ * for convenience.
+ */
+
+/**
+ * struct bcm2835_isp_rational - Rational value type.
+ *
+ * @num: Numerator.
+ * @den: Denominator.
+ */
+struct bcm2835_isp_rational {
+ __s32 num;
+ __s32 den;
+};
+
+/**
+ * struct bcm2835_isp_ccm - Colour correction matrix.
+ *
+ * @ccm: 3x3 correction matrix coefficients.
+ * @offsets: 1x3 correction offsets.
+ */
+struct bcm2835_isp_ccm {
+ struct bcm2835_isp_rational ccm[3][3];
+ __s32 offsets[3];
+};
+
+/**
+ * struct bcm2835_isp_custom_ccm - Custom CCM applied with the
+ * V4L2_CID_USER_BCM2835_ISP_CC_MATRIX ctrl.
+ *
+ * @enabled: Enable custom CCM.
+ * @ccm: Custom CCM coefficients and offsets.
+ */
+struct bcm2835_isp_custom_ccm {
+ __u32 enabled;
+ struct bcm2835_isp_ccm ccm;
+};
+
+/**
+ * enum bcm2835_isp_gain_format - format of the gains in the lens shading
+ * tables used with the
+ * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING ctrl.
+ *
+ * @GAIN_FORMAT_U0P8_1: Gains are u0.8 format, starting at 1.0
+ * @GAIN_FORMAT_U1P7_0: Gains are u1.7 format, starting at 0.0
+ * @GAIN_FORMAT_U1P7_1: Gains are u1.7 format, starting at 1.0
+ * @GAIN_FORMAT_U2P6_0: Gains are u2.6 format, starting at 0.0
+ * @GAIN_FORMAT_U2P6_1: Gains are u2.6 format, starting at 1.0
+ * @GAIN_FORMAT_U3P5_0: Gains are u3.5 format, starting at 0.0
+ * @GAIN_FORMAT_U3P5_1: Gains are u3.5 format, starting at 1.0
+ * @GAIN_FORMAT_U4P10: Gains are u4.10 format, starting at 0.0
+ */
+enum bcm2835_isp_gain_format {
+ GAIN_FORMAT_U0P8_1 = 0,
+ GAIN_FORMAT_U1P7_0 = 1,
+ GAIN_FORMAT_U1P7_1 = 2,
+ GAIN_FORMAT_U2P6_0 = 3,
+ GAIN_FORMAT_U2P6_1 = 4,
+ GAIN_FORMAT_U3P5_0 = 5,
+ GAIN_FORMAT_U3P5_1 = 6,
+ GAIN_FORMAT_U4P10 = 7,
+};
+
+/**
+ * struct bcm2835_isp_lens_shading - Lens shading tables supplied with the
+ * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING
+ * ctrl.
+ *
+ * @enabled: Enable lens shading.
+ * @grid_cell_size: Size of grid cells in samples (16, 32, 64, 128 or 256).
+ * @grid_width: Width of lens shading tables in grid cells.
+ * @grid_stride: Row to row distance (in grid cells) between grid cells
+ * in the same horizontal location.
+ * @grid_height: Height of lens shading tables in grid cells.
+ * @mem_handle_table: Memory handle to the tables.
+ * @ref_transform: Reference transform - unsupported, please pass zero.
+ * @corner_sampled: Whether the gains are sampled at the corner points
+ * of the grid cells or in the cell centres.
+ * @gain_format: Format of the gains (see enum &bcm2835_isp_gain_format).
+ */
+struct bcm2835_isp_lens_shading {
+ __u32 enabled;
+ __u32 grid_cell_size;
+ __u32 grid_width;
+ __u32 grid_stride;
+ __u32 grid_height;
+ __u32 mem_handle_table;
+ __u32 ref_transform;
+ __u32 corner_sampled;
+ __u32 gain_format;
+};
+
+/**
+ * struct bcm2835_isp_black_level - Sensor black level set with the
+ * V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL ctrl.
+ *
+ * @enabled: Enable black level.
+ * @black_level_r: Black level for red channel.
+ * @black_level_g: Black level for green channels.
+ * @black_level_b: Black level for blue channel.
+ */
+struct bcm2835_isp_black_level {
+ __u32 enabled;
+ __u16 black_level_r;
+ __u16 black_level_g;
+ __u16 black_level_b;
+ __u8 pad_[2]; /* Unused */
+};
+
+/**
+ * struct bcm2835_isp_geq - Green equalisation parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_GEQ ctrl.
+ *
+ * @enabled: Enable green equalisation.
+ * @offset: Fixed offset of the green equalisation threshold.
+ * @slope: Slope of the green equalisation threshold.
+ */
+struct bcm2835_isp_geq {
+ __u32 enabled;
+ __u32 offset;
+ struct bcm2835_isp_rational slope;
+};
+
+#define BCM2835_NUM_GAMMA_PTS 33
+
+/**
+ * struct bcm2835_isp_gamma - Gamma parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_GAMMA ctrl.
+ *
+ * @enabled: Enable gamma adjustment.
+ * @X: X values of the points defining the gamma curve.
+ * Values should be scaled to 16 bits.
+ * @Y: Y values of the points defining the gamma curve.
+ * Values should be scaled to 16 bits.
+ */
+struct bcm2835_isp_gamma {
+ __u32 enabled;
+ __u16 x[BCM2835_NUM_GAMMA_PTS];
+ __u16 y[BCM2835_NUM_GAMMA_PTS];
+};
+
+/**
+ * struct bcm2835_isp_denoise - Denoise parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_DENOISE ctrl.
+ *
+ * @enabled: Enable denoise.
+ * @constant: Fixed offset of the noise threshold.
+ * @slope: Slope of the noise threshold.
+ * @strength: Denoise strength between 0.0 (off) and 1.0 (maximum).
+ */
+struct bcm2835_isp_denoise {
+ __u32 enabled;
+ __u32 constant;
+ struct bcm2835_isp_rational slope;
+ struct bcm2835_isp_rational strength;
+};
+
+/**
+ * struct bcm2835_isp_sharpen - Sharpen parameters set with the
+ * V4L2_CID_USER_BCM2835_ISP_SHARPEN ctrl.
+ *
+ * @enabled: Enable sharpening.
+ * @threshold: Threshold at which to start sharpening pixels.
+ * @strength: Strength with which pixel sharpening increases.
+ * @limit: Limit to the amount of sharpening applied.
+ */
+struct bcm2835_isp_sharpen {
+ __u32 enabled;
+ struct bcm2835_isp_rational threshold;
+ struct bcm2835_isp_rational strength;
+ struct bcm2835_isp_rational limit;
+};
+
+/**
+ * enum bcm2835_isp_dpc_mode - defective pixel correction (DPC) strength.
+ *
+ * @DPC_MODE_OFF: No DPC.
+ * @DPC_MODE_NORMAL: Normal DPC.
+ * @DPC_MODE_STRONG: Strong DPC.
+ */
+enum bcm2835_isp_dpc_mode {
+ DPC_MODE_OFF = 0,
+ DPC_MODE_NORMAL = 1,
+ DPC_MODE_STRONG = 2,
+};
+
+/**
+ * struct bcm2835_isp_dpc - Defective pixel correction (DPC) parameters set
+ * with the V4L2_CID_USER_BCM2835_ISP_DPC ctrl.
+ *
+ * @enabled: Enable DPC.
+ * @strength: DPC strength (see enum &bcm2835_isp_dpc_mode).
+ */
+struct bcm2835_isp_dpc {
+ __u32 enabled;
+ __u32 strength;
+};
+
+/*
+ * ISP statistics structures.
+ *
+ * The bcm2835_isp_stats structure is generated at the output of the
+ * statistics node. Note that this does not directly map onto the statistics
+ * output of the ISP HW. Instead, the MMAL firmware code maps the HW statistics
+ * to the bcm2835_isp_stats structure.
+ */
+#define DEFAULT_AWB_REGIONS_X 16
+#define DEFAULT_AWB_REGIONS_Y 12
+
+#define NUM_HISTOGRAMS 2
+#define NUM_HISTOGRAM_BINS 128
+#define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
+#define FLOATING_REGIONS 16
+#define AGC_REGIONS 16
+#define FOCUS_REGIONS 12
+
+/**
+ * struct bcm2835_isp_stats_hist - Histogram statistics
+ *
+ * @r_hist: Red channel histogram.
+ * @g_hist: Combined green channel histogram.
+ * @b_hist: Blue channel histogram.
+ */
+struct bcm2835_isp_stats_hist {
+ __u32 r_hist[NUM_HISTOGRAM_BINS];
+ __u32 g_hist[NUM_HISTOGRAM_BINS];
+ __u32 b_hist[NUM_HISTOGRAM_BINS];
+};
+
+/**
+ * struct bcm2835_isp_stats_region - Region sums.
+ *
+ * @counted: The number of 2x2 bayer tiles accumulated.
+ * @notcounted: The number of 2x2 bayer tiles not accumulated.
+ * @r_sum: Total sum of counted pixels in the red channel for a region.
+ * @g_sum: Total sum of counted pixels in the green channel for a region.
+ * @b_sum: Total sum of counted pixels in the blue channel for a region.
+ */
+struct bcm2835_isp_stats_region {
+ __u32 counted;
+ __u32 notcounted;
+ __u64 r_sum;
+ __u64 g_sum;
+ __u64 b_sum;
+};
+
+/**
+ * struct bcm2835_isp_stats_focus - Focus statistics.
+ *
+ * @contrast_val: Focus measure - accumulated output of the focus filter.
+ * In the first dimension, index [0] counts pixels below a
+ * preset threshold, and index [1] counts pixels above the
+ * threshold. In the second dimension, index [0] uses the
+ * first predefined filter, and index [1] uses the second
+ * predefined filter.
+ * @contrast_val_num: The number of counted pixels in the above accumulation.
+ */
+struct bcm2835_isp_stats_focus {
+ __u64 contrast_val[2][2];
+ __u32 contrast_val_num[2][2];
+};
+
+/**
+ * struct bcm2835_isp_stats - ISP statistics.
+ *
+ * @version: Version of the bcm2835_isp_stats structure.
+ * @size: Size of the bcm2835_isp_stats structure.
+ * @hist: Histogram statistics for the entire image.
+ * @awb_stats: Statistics for the regions defined for AWB calculations.
+ * @floating_stats: Statistics for arbitrarily placed (floating) regions.
+ * @agc_stats: Statistics for the regions defined for AGC calculations.
+ * @focus_stats: Focus filter statistics for the focus regions.
+ */
+struct bcm2835_isp_stats {
+ __u32 version;
+ __u32 size;
+ struct bcm2835_isp_stats_hist hist[NUM_HISTOGRAMS];
+ struct bcm2835_isp_stats_region awb_stats[AWB_REGIONS];
+ struct bcm2835_isp_stats_region floating_stats[FLOATING_REGIONS];
+ struct bcm2835_isp_stats_region agc_stats[AGC_REGIONS];
+ struct bcm2835_isp_stats_focus focus_stats[FOCUS_REGIONS];
+};
+
+#endif /* __BCM2835_ISP_H_ */
diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
index 106f71e709df..072f3c755a68 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
+++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
@@ -5,4 +5,5 @@ config BCM2835_VCHIQ_MMAL
help
Enables the MMAL API over VCHIQ interface as used for the
majority of the multimedia services on VideoCore.
- Defaults to Y when the Broadcomd BCM2835 camera host is selected.
+ Defaults to Y when the Broadcomd BCM2835 camera host or ISP are
+ selected.
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
index 44ba91aa6d47..8d904fcce388 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
@@ -100,6 +100,10 @@
*/
#define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
+/** ISP image statistics format
+ */
+#define MMAL_ENCODING_BRCM_STATS MMAL_FOURCC('S', 'T', 'A', 'T')
+
/* }@ */
/** \name Pre-defined audio encodings */
diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
index 1793103b18fd..b3552af5cf8f 100644
--- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
+++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
@@ -221,6 +221,62 @@ enum mmal_parameter_camera_type {
MMAL_PARAMETER_SHUTTER_SPEED,
/**< Takes a @ref MMAL_PARAMETER_AWB_GAINS_T */
MMAL_PARAMETER_CUSTOM_AWB_GAINS,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_SETTINGS_T */
+ MMAL_PARAMETER_CAMERA_SETTINGS,
+ /**< Takes a @ref MMAL_PARAMETER_PRIVACY_INDICATOR_T */
+ MMAL_PARAMETER_PRIVACY_INDICATOR,
+ /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
+ MMAL_PARAMETER_VIDEO_DENOISE,
+ /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
+ MMAL_PARAMETER_STILLS_DENOISE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_ANNOTATE_T */
+ MMAL_PARAMETER_ANNOTATE,
+ /**< Takes a @ref MMAL_PARAMETER_STEREOSCOPIC_MODE_T */
+ MMAL_PARAMETER_STEREOSCOPIC_MODE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_INTERFACE_T */
+ MMAL_PARAMETER_CAMERA_INTERFACE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_CLOCKING_MODE_T */
+ MMAL_PARAMETER_CAMERA_CLOCKING_MODE,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_CONFIG_T */
+ MMAL_PARAMETER_CAMERA_RX_CONFIG,
+ /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_TIMING_T */
+ MMAL_PARAMETER_CAMERA_RX_TIMING,
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_DPF_CONFIG,
+
+ /* 0x50 */
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_JPEG_RESTART_INTERVAL,
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_CAMERA_ISP_BLOCK_OVERRIDE,
+ /**< Takes a @ref MMAL_PARAMETER_LENS_SHADING_T */
+ MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
+ /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
+ MMAL_PARAMETER_BLACK_LEVEL,
+ /**< Takes a @ref MMAL_PARAMETER_RESIZE_T */
+ MMAL_PARAMETER_RESIZE_PARAMS,
+ /**< Takes a @ref MMAL_PARAMETER_CROP_T */
+ MMAL_PARAMETER_CROP,
+ /**< Takes a @ref MMAL_PARAMETER_INT32_T */
+ MMAL_PARAMETER_OUTPUT_SHIFT,
+ /**< Takes a @ref MMAL_PARAMETER_INT32_T */
+ MMAL_PARAMETER_CCM_SHIFT,
+ /**< Takes a @ref MMAL_PARAMETER_CUSTOM_CCM_T */
+ MMAL_PARAMETER_CUSTOM_CCM,
+ /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
+ MMAL_PARAMETER_ANALOG_GAIN,
+ /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
+ MMAL_PARAMETER_DIGITAL_GAIN,
+ /**< Takes a @ref MMAL_PARAMETER_DENOISE_T */
+ MMAL_PARAMETER_DENOISE,
+ /**< Takes a @ref MMAL_PARAMETER_SHARPEN_T */
+ MMAL_PARAMETER_SHARPEN,
+ /**< Takes a @ref MMAL_PARAMETER_GEQ_T */
+ MMAL_PARAMETER_GEQ,
+ /**< Tales a @ref MMAP_PARAMETER_DPC_T */
+ MMAL_PARAMETER_DPC,
+ /**< Tales a @ref MMAP_PARAMETER_GAMMA_T */
+ MMAL_PARAMETER_GAMMA,
};
struct mmal_parameter_rational {
@@ -779,7 +835,102 @@ struct mmal_parameter_camera_info {
struct mmal_parameter_camera_info_camera
cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
struct mmal_parameter_camera_info_flash
- flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
+ flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
+};
+
+struct mmal_parameter_ccm {
+ struct mmal_parameter_rational ccm[3][3];
+ s32 offsets[3];
+};
+
+struct mmal_parameter_custom_ccm {
+ u32 enabled; /**< Enable the custom CCM. */
+ struct mmal_parameter_ccm ccm; /**< CCM to be used. */
+};
+
+struct mmal_parameter_lens_shading {
+ u32 enabled;
+ u32 grid_cell_size;
+ u32 grid_width;
+ u32 grid_stride;
+ u32 grid_height;
+ u32 mem_handle_table;
+ u32 ref_transform;
+};
+
+enum mmal_parameter_ls_gain_format_type {
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U0P8_1 = 0,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_0 = 1,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_1 = 2,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_0 = 3,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_1 = 4,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_0 = 5,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_1 = 6,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U4P10 = 7,
+ MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_DUMMY = 0x7FFFFFFF
+};
+
+struct mmal_parameter_lens_shading_v2 {
+ u32 enabled;
+ u32 grid_cell_size;
+ u32 grid_width;
+ u32 grid_stride;
+ u32 grid_height;
+ u32 mem_handle_table;
+ u32 ref_transform;
+ u32 corner_sampled;
+ enum mmal_parameter_ls_gain_format_type gain_format;
+};
+
+struct mmal_parameter_black_level {
+ u32 enabled;
+ u16 black_level_r;
+ u16 black_level_g;
+ u16 black_level_b;
+ u8 pad_[2]; /* Unused */
+};
+
+struct mmal_parameter_geq {
+ u32 enabled;
+ u32 offset;
+ struct mmal_parameter_rational slope;
+};
+
+#define MMAL_NUM_GAMMA_PTS 33
+struct mmal_parameter_gamma {
+ u32 enabled;
+ u16 x[MMAL_NUM_GAMMA_PTS];
+ u16 y[MMAL_NUM_GAMMA_PTS];
+};
+
+struct mmal_parameter_denoise {
+ u32 enabled;
+ u32 constant;
+ struct mmal_parameter_rational slope;
+ struct mmal_parameter_rational strength;
+};
+
+struct mmal_parameter_sharpen {
+ u32 enabled;
+ struct mmal_parameter_rational threshold;
+ struct mmal_parameter_rational strength;
+ struct mmal_parameter_rational limit;
+};
+
+enum mmal_dpc_mode {
+ MMAL_DPC_MODE_OFF = 0,
+ MMAL_DPC_MODE_NORMAL = 1,
+ MMAL_DPC_MODE_STRONG = 2,
+ MMAL_DPC_MODE_MAX = 0x7FFFFFFF,
+};
+
+struct mmal_parameter_dpc {
+ u32 enabled;
+ u32 strength;
+};
+
+struct mmal_parameter_crop {
+ struct vchiq_mmal_rect rect;
};
#endif
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-04 9:26 ` [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP Laurent Pinchart
@ 2020-05-11 19:19 ` Nicolas Saenz Julienne
2020-05-18 13:38 ` Dave Stevenson
2020-05-18 12:02 ` Hans Verkuil
1 sibling, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-11 19:19 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]
Hi Naushir,
a small comment.
On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
> chips.
>
> The driver interfaces though the VideoCore unit using the VCHIQ MMAL
> interface.
>
> ISP driver upported from from RaspberryPi BSP at revision:
> 6c3505be6c3e ("staging: vc04_services: isp: Make all references to
> bcm2835_isp_fmt const")
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> [Adapt to staging by moving all modifications that in the BSP are scattered
> in core components inside this directory]
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
[...]
> +static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
> +{
> + mmal_vchi_buffer_cleanup(mmal_buf);
> +
> + if (mmal_buf->dma_buf) {
> + dma_buf_put(mmal_buf->dma_buf);
> + mmal_buf->dma_buf = NULL;
Why is this needed here, shouldn't this be mmal-vchi's responsibility? IIUC the
original dma_buf_get() happens there too.
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-11 19:19 ` Nicolas Saenz Julienne
@ 2020-05-18 13:38 ` Dave Stevenson
2020-05-20 13:46 ` Nicolas Saenz Julienne
0 siblings, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-05-18 13:38 UTC (permalink / raw)
To: Nicolas Saenz Julienne
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
Hi Nicholas
Sorry for the slight delay in replying.
On Mon, 11 May 2020 at 20:19, Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Hi Naushir,
> a small comment.
>
> On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
> > chips.
> >
> > The driver interfaces though the VideoCore unit using the VCHIQ MMAL
> > interface.
> >
> > ISP driver upported from from RaspberryPi BSP at revision:
> > 6c3505be6c3e ("staging: vc04_services: isp: Make all references to
> > bcm2835_isp_fmt const")
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > [Adapt to staging by moving all modifications that in the BSP are scattered
> > in core components inside this directory]
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
>
> [...]
>
> > +static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
> > +{
> > + mmal_vchi_buffer_cleanup(mmal_buf);
> > +
> > + if (mmal_buf->dma_buf) {
> > + dma_buf_put(mmal_buf->dma_buf);
> > + mmal_buf->dma_buf = NULL;
>
> Why is this needed here, shouldn't this be mmal-vchi's responsibility? IIUC the
> original dma_buf_get() happens there too.
The original dma_buf_get is in bcm2835_isp_buf_prepare as it either
comes from a dma_buf_get(vb->planes[0].m.fd) for VB2_MEMORY_DMABUF
operations, or a vb2_core_expbuf_dmabuf for VB2_MEMORY_MMAP.
Seeing as the original "get" is in this calling code, it seemed
reasonable that the put is as well.
There are no get/put operations on dma-bufs (other than indirectly
through vc_sm_cma) within mmal-vchiq. You have the call
vc_sm_cma_import_dmabuf to take the external dma_buf and pull it into
vc_sm_cma, but that is the limit of it.
Dave
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-18 13:38 ` Dave Stevenson
@ 2020-05-20 13:46 ` Nicolas Saenz Julienne
0 siblings, 0 replies; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-20 13:46 UTC (permalink / raw)
To: Dave Stevenson
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
[-- Attachment #1: Type: text/plain, Size: 2144 bytes --]
On Mon, 2020-05-18 at 14:38 +0100, Dave Stevenson wrote:
> Hi Nicholas
>
> Sorry for the slight delay in replying.
>
> On Mon, 11 May 2020 at 20:19, Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
> > Hi Naushir,
> > a small comment.
> >
> > On Mon, 2020-05-04 at 12:26 +0300, Laurent Pinchart wrote:
> > > From: Naushir Patuck <naush@raspberrypi.com>
> > >
> > > Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
> > > chips.
> > >
> > > The driver interfaces though the VideoCore unit using the VCHIQ MMAL
> > > interface.
> > >
> > > ISP driver upported from from RaspberryPi BSP at revision:
> > > 6c3505be6c3e ("staging: vc04_services: isp: Make all references to
> > > bcm2835_isp_fmt const")
> > >
> > > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > > [Adapt to staging by moving all modifications that in the BSP are
> > > scattered
> > > in core components inside this directory]
> > > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > > ---
> >
> > [...]
> >
> > > +static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
> > > +{
> > > + mmal_vchi_buffer_cleanup(mmal_buf);
> > > +
> > > + if (mmal_buf->dma_buf) {
> > > + dma_buf_put(mmal_buf->dma_buf);
> > > + mmal_buf->dma_buf = NULL;
> >
> > Why is this needed here, shouldn't this be mmal-vchi's responsibility? IIUC
> > the
> > original dma_buf_get() happens there too.
>
> The original dma_buf_get is in bcm2835_isp_buf_prepare as it either
> comes from a dma_buf_get(vb->planes[0].m.fd) for VB2_MEMORY_DMABUF
> operations, or a vb2_core_expbuf_dmabuf for VB2_MEMORY_MMAP.
> Seeing as the original "get" is in this calling code, it seemed
> reasonable that the put is as well.
>
> There are no get/put operations on dma-bufs (other than indirectly
> through vc_sm_cma) within mmal-vchiq. You have the call
> vc_sm_cma_import_dmabuf to take the external dma_buf and pull it into
> vc_sm_cma, but that is the limit of it.
Understood, thanks.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-04 9:26 ` [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP Laurent Pinchart
2020-05-11 19:19 ` Nicolas Saenz Julienne
@ 2020-05-18 12:02 ` Hans Verkuil
2020-05-18 14:36 ` Dave Stevenson
2020-05-19 12:47 ` Naushir Patuck
1 sibling, 2 replies; 111+ messages in thread
From: Hans Verkuil @ 2020-05-18 12:02 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
On 04/05/2020 11:26, Laurent Pinchart wrote:
> From: Naushir Patuck <naush@raspberrypi.com>
>
> Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
> chips.
>
> The driver interfaces though the VideoCore unit using the VCHIQ MMAL
> interface.
>
> ISP driver upported from from RaspberryPi BSP at revision:
> 6c3505be6c3e ("staging: vc04_services: isp: Make all references to bcm2835_isp_fmt const")
>
> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> [Adapt to staging by moving all modifications that in the BSP are scattered
> in core components inside this directory]
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 41 +
> drivers/staging/vc04_services/Kconfig | 2 +
> drivers/staging/vc04_services/Makefile | 1 +
> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> .../vc04_services/bcm2835-isp/Makefile | 10 +
> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1632 +++++++++++++++++
> .../bcm2835-isp/bcm2835_isp_ctrls.h | 67 +
> .../bcm2835-isp/bcm2835_isp_fmts.h | 301 +++
> .../include/uapi/linux/bcm2835-isp.h | 333 ++++
> .../staging/vc04_services/vchiq-mmal/Kconfig | 3 +-
> .../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
> .../vchiq-mmal/mmal-parameters.h | 153 +-
> 12 files changed, 2559 insertions(+), 2 deletions(-)
> create mode 100644 drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> create mode 100644 drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
>
<snip>
> diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> new file mode 100644
> index 000000000000..a32faab4b8dc
> --- /dev/null
> +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> @@ -0,0 +1,1632 @@
<snip>
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Broadcom BCM2835 ISP driver
> + *
> + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> + *
> + * Author: Naushir Patuck (naush@raspberrypi.com)
> + *
> + */
> +
> +#include <linux/module.h>
> +#include <linux/platform_device.h>
> +
> +#include <media/v4l2-ctrls.h>
> +#include <media/v4l2-device.h>
> +#include <media/v4l2-event.h>
> +#include <media/v4l2-ioctl.h>
> +#include <media/videobuf2-dma-contig.h>
> +
> +#include "vchiq-mmal/mmal-msg.h"
> +#include "vchiq-mmal/mmal-parameters.h"
> +#include "vchiq-mmal/mmal-vchiq.h"
> +
> +#include "bcm2835_isp_ctrls.h"
> +#include "bcm2835_isp_fmts.h"
> +
> +static unsigned int debug;
> +module_param(debug, uint, 0644);
> +MODULE_PARM_DESC(debug, "activates debug info");
> +
> +static unsigned int video_nr = 13;
> +module_param(video_nr, uint, 0644);
> +MODULE_PARM_DESC(video_nr, "base video device number");
> +
> +#define BCM2835_ISP_NAME "bcm2835-isp"
> +#define BCM2835_ISP_ENTITY_NAME_LEN 32
> +
> +#define BCM2835_ISP_NUM_OUTPUTS 1
> +#define BCM2835_ISP_NUM_CAPTURES 2
> +#define BCM2835_ISP_NUM_METADATA 1
> +
> +#define BCM2835_ISP_NUM_NODES \
> + (BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES + \
> + BCM2835_ISP_NUM_METADATA)
> +
> +/* Default frame dimension of 1280 pixels. */
> +#define DEFAULT_DIM 1280U
> +/*
> + * Maximum frame dimension of 16384 pixels. Even though the ISP runs in tiles,
> + * have a sensible limit so that we do not create an excessive number of tiles
> + * to process.
> + */
> +#define MAX_DIM 16384U
> +/*
> + * Minimum frame dimension of 64 pixels. Anything lower, and the tiling
> + * algorihtm may not be able to cope when applying filter context.
> + */
> +#define MIN_DIM 64U
> +
> +/* Per-queue, driver-specific private data */
> +struct bcm2835_isp_q_data {
> + /*
> + * These parameters should be treated as gospel, with everything else
> + * being determined from them.
> + */
> + unsigned int bytesperline;
> + unsigned int width;
> + unsigned int height;
> + unsigned int sizeimage;
> + const struct bcm2835_isp_fmt *fmt;
> +};
> +
> +/*
> + * Structure to describe a single node /dev/video<N> which represents a single
> + * input or output queue to the ISP device.
> + */
> +struct bcm2835_isp_node {
> + int vfl_dir;
> + unsigned int id;
> + const char *name;
> + struct video_device vfd;
> + struct media_pad pad;
> + struct media_intf_devnode *intf_devnode;
> + struct media_link *intf_link;
> + struct mutex lock; /* top level device node lock */
> + struct mutex queue_lock;
> +
> + struct vb2_queue queue;
> + unsigned int sequence;
> +
> + /* The list of formats supported on the node. */
> + struct bcm2835_isp_fmt_list supported_fmts;
> +
> + struct bcm2835_isp_q_data q_data;
> +
> + /* Parent device structure */
> + struct bcm2835_isp_dev *dev;
> +
> + bool registered;
> + bool media_node_registered;
> + bool queue_init;
> +};
> +
> +/*
> + * Structure representing the entire ISP device, comprising several input and
> + * output nodes /dev/video<N>.
> + */
> +struct bcm2835_isp_dev {
> + struct v4l2_device v4l2_dev;
> + struct device *dev;
> + struct v4l2_ctrl_handler ctrl_handler;
> + struct media_device mdev;
> + struct media_entity entity;
> + bool media_device_registered;
> + bool media_entity_registered;
> + struct vchiq_mmal_instance *mmal_instance;
> + struct vchiq_mmal_component *component;
> + struct completion frame_cmplt;
> +
> + struct bcm2835_isp_node node[BCM2835_ISP_NUM_NODES];
> + struct media_pad pad[BCM2835_ISP_NUM_NODES];
> + atomic_t num_streaming;
> +
> + /* Image pipeline controls. */
> + int r_gain;
> + int b_gain;
> +};
> +
> +struct bcm2835_isp_buffer {
> + struct vb2_v4l2_buffer vb;
> + struct mmal_buffer mmal;
> +};
> +
> +static
> +inline struct bcm2835_isp_dev *node_get_dev(struct bcm2835_isp_node *node)
> +{
> + return node->dev;
> +}
> +
> +static inline bool node_is_output(struct bcm2835_isp_node *node)
> +{
> + return node->queue.type == V4L2_BUF_TYPE_VIDEO_OUTPUT;
> +}
> +
> +static inline bool node_is_capture(struct bcm2835_isp_node *node)
> +{
> + return node->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +}
> +
> +static inline bool node_is_stats(struct bcm2835_isp_node *node)
> +{
> + return node->queue.type == V4L2_BUF_TYPE_META_CAPTURE;
> +}
> +
> +static inline enum v4l2_buf_type index_to_queue_type(int index)
> +{
> + if (index < BCM2835_ISP_NUM_OUTPUTS)
> + return V4L2_BUF_TYPE_VIDEO_OUTPUT;
> + else if (index < BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES)
> + return V4L2_BUF_TYPE_VIDEO_CAPTURE;
> + else
> + return V4L2_BUF_TYPE_META_CAPTURE;
> +}
> +
> +static struct vchiq_mmal_port *get_port_data(struct bcm2835_isp_node *node)
> +{
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> +
> + if (!dev->component)
> + return NULL;
> +
> + switch (node->queue.type) {
> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> + return &dev->component->input[node->id];
> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> + case V4L2_BUF_TYPE_META_CAPTURE:
> + return &dev->component->output[node->id];
> + default:
> + v4l2_err(&dev->v4l2_dev, "%s: Invalid queue type %u\n",
> + __func__, node->queue.type);
> + break;
> + }
> + return NULL;
> +}
> +
> +static int set_isp_param(struct bcm2835_isp_node *node, u32 parameter,
> + void *value, u32 value_size)
> +{
> + struct vchiq_mmal_port *port = get_port_data(node);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> +
> + return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> + parameter, value, value_size);
> +}
> +
> +static int set_wb_gains(struct bcm2835_isp_node *node)
> +{
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct mmal_parameter_awbgains gains = {
> + .r_gain = { dev->r_gain, 1000 },
> + .b_gain = { dev->b_gain, 1000 }
> + };
> +
> + return set_isp_param(node, MMAL_PARAMETER_CUSTOM_AWB_GAINS,
> + &gains, sizeof(gains));
> +}
> +
> +static int set_digital_gain(struct bcm2835_isp_node *node, uint32_t gain)
> +{
> + struct mmal_parameter_rational digital_gain = {
> + .num = gain,
> + .den = 1000
> + };
> +
> + return set_isp_param(node, MMAL_PARAMETER_DIGITAL_GAIN,
> + &digital_gain, sizeof(digital_gain));
> +}
> +
> +static const struct bcm2835_isp_fmt *get_fmt(u32 mmal_fmt)
> +{
> + unsigned int i;
> +
> + for (i = 0; i < ARRAY_SIZE(supported_formats); i++) {
> + if (supported_formats[i].mmal_fmt == mmal_fmt)
> + return &supported_formats[i];
> + }
> + return NULL;
> +}
> +
> +static const
> +struct bcm2835_isp_fmt *find_format_by_fourcc(unsigned int fourcc,
> + struct bcm2835_isp_node *node)
> +{
> + struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
> + const struct bcm2835_isp_fmt *fmt;
> + unsigned int i;
> +
> + for (i = 0; i < fmts->num_entries; i++) {
> + fmt = fmts->list[i];
> + if (fmt->fourcc == fourcc)
> + return fmt;
> + }
> +
> + return NULL;
> +}
> +
> +static const
> +struct bcm2835_isp_fmt *find_format(struct v4l2_format *f,
> + struct bcm2835_isp_node *node)
> +{
> + return find_format_by_fourcc(node_is_stats(node) ?
> + f->fmt.meta.dataformat :
> + f->fmt.pix.pixelformat,
> + node);
> +}
> +
> +/* vb2_to_mmal_buffer() - converts vb2 buffer header to MMAL
> + *
> + * Copies all the required fields from a VB2 buffer to the MMAL buffer header,
> + * ready for sending to the VPU.
> + */
> +static void vb2_to_mmal_buffer(struct mmal_buffer *buf,
> + struct vb2_v4l2_buffer *vb2)
> +{
> + u64 pts;
> +
> + buf->mmal_flags = 0;
> + if (vb2->flags & V4L2_BUF_FLAG_KEYFRAME)
> + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_KEYFRAME;
> +
> + /* Data must be framed correctly as one frame per buffer. */
> + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_FRAME_END;
> +
> + buf->length = vb2->vb2_buf.planes[0].bytesused;
> + /*
> + * Minor ambiguity in the V4L2 spec as to whether passing in a 0 length
> + * buffer, or one with V4L2_BUF_FLAG_LAST set denotes end of stream.
> + * Handle either.
> + */
> + if (!buf->length || vb2->flags & V4L2_BUF_FLAG_LAST)
> + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_EOS;
> +
> + /* vb2 timestamps in nsecs, mmal in usecs */
> + pts = vb2->vb2_buf.timestamp;
> + do_div(pts, 1000);
> + buf->pts = pts;
> + buf->dts = MMAL_TIME_UNKNOWN;
> +}
> +
> +static void mmal_buffer_cb(struct vchiq_mmal_instance *instance,
> + struct vchiq_mmal_port *port, int status,
> + struct mmal_buffer *mmal_buf)
> +{
> + struct bcm2835_isp_buffer *q_buf;
> + struct bcm2835_isp_node *node = port->cb_ctx;
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct vb2_v4l2_buffer *vb2;
> +
> + q_buf = container_of(mmal_buf, struct bcm2835_isp_buffer, mmal);
> + vb2 = &q_buf->vb;
> + v4l2_dbg(2, debug, &dev->v4l2_dev,
> + "%s: port:%s[%d], status:%d, buf:%p, dmabuf:%p, length:%lu, flags %u, pts %lld\n",
> + __func__, node_is_output(node) ? "input" : "output", node->id,
> + status, mmal_buf, mmal_buf->dma_buf, mmal_buf->length,
> + mmal_buf->mmal_flags, mmal_buf->pts);
> +
> + if (mmal_buf->cmd)
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Unexpected event on output callback - %08x\n",
> + __func__, mmal_buf->cmd);
> +
> + if (status) {
> + /* error in transfer */
> + if (vb2) {
> + /* there was a buffer with the error so return it */
> + vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_ERROR);
> + }
> + return;
> + }
> +
> + /* vb2 timestamps in nsecs, mmal in usecs */
> + vb2->vb2_buf.timestamp = mmal_buf->pts * 1000;
> + vb2->sequence = node->sequence++;
> + vb2_set_plane_payload(&vb2->vb2_buf, 0, mmal_buf->length);
> + vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_DONE);
> +
> + if (!port->enabled)
> + complete(&dev->frame_cmplt);
> +}
> +
> +static void setup_mmal_port_format(struct bcm2835_isp_node *node,
> + struct vchiq_mmal_port *port)
> +{
> + struct bcm2835_isp_q_data *q_data = &node->q_data;
> +
> + port->format.encoding = q_data->fmt->mmal_fmt;
> + /* Raw image format - set width/height */
> + port->es.video.width = (q_data->bytesperline << 3) / q_data->fmt->depth;
> + port->es.video.height = q_data->height;
> + port->es.video.crop.width = q_data->width;
> + port->es.video.crop.height = q_data->height;
> + port->es.video.crop.x = 0;
> + port->es.video.crop.y = 0;
> +};
> +
> +static int setup_mmal_port(struct bcm2835_isp_node *node)
> +{
> + struct vchiq_mmal_port *port = get_port_data(node);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + unsigned int enable = 1;
> + int ret;
> +
> + v4l2_dbg(2, debug, &dev->v4l2_dev, "%s: setup %s[%d]\n", __func__,
> + node->name, node->id);
> +
> + vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> + MMAL_PARAMETER_ZERO_COPY, &enable,
> + sizeof(enable));
> + setup_mmal_port_format(node, port);
> + ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
> + if (ret < 0) {
> + v4l2_dbg(1, debug, &dev->v4l2_dev,
> + "%s: vchiq_mmal_port_set_format failed\n",
> + __func__);
> + return ret;
> + }
> +
> + if (node->q_data.sizeimage < port->minimum_buffer.size) {
> + v4l2_err(&dev->v4l2_dev,
> + "buffer size mismatch sizeimage %u < min size %u\n",
> + node->q_data.sizeimage, port->minimum_buffer.size);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
> +{
> + mmal_vchi_buffer_cleanup(mmal_buf);
> +
> + if (mmal_buf->dma_buf) {
> + dma_buf_put(mmal_buf->dma_buf);
> + mmal_buf->dma_buf = NULL;
> + }
> +
> + return 0;
> +}
> +
> +static int bcm2835_isp_node_queue_setup(struct vb2_queue *q,
> + unsigned int *nbuffers,
> + unsigned int *nplanes,
> + unsigned int sizes[],
> + struct device *alloc_devs[])
> +{
> + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> + struct vchiq_mmal_port *port;
> + unsigned int size;
> +
> + if (setup_mmal_port(node))
> + return -EINVAL;
> +
> + size = node->q_data.sizeimage;
> + if (size == 0) {
> + v4l2_info(&node_get_dev(node)->v4l2_dev,
> + "%s: Image size unset in queue_setup for node %s[%d]\n",
> + __func__, node->name, node->id);
> + return -EINVAL;
> + }
> +
> + if (*nplanes)
> + return sizes[0] < size ? -EINVAL : 0;
> +
> + *nplanes = 1;
> + sizes[0] = size;
> +
> + port = get_port_data(node);
> + port->current_buffer.size = size;
> +
> + if (*nbuffers < port->minimum_buffer.num)
> + *nbuffers = port->minimum_buffer.num;
> +
> + port->current_buffer.num = *nbuffers;
> +
> + v4l2_dbg(2, debug, &node_get_dev(node)->v4l2_dev,
> + "%s: Image size %u, nbuffers %u for node %s[%d]\n",
> + __func__, sizes[0], *nbuffers, node->name, node->id);
> + return 0;
> +}
> +
> +static int bcm2835_isp_buf_init(struct vb2_buffer *vb)
> +{
> + struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> + struct bcm2835_isp_buffer *buf =
> + container_of(vb2, struct bcm2835_isp_buffer, vb);
> +
> + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: vb %p\n", __func__, vb);
> +
> + buf->mmal.buffer = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
> + buf->mmal.buffer_size = vb2_plane_size(&buf->vb.vb2_buf, 0);
> + mmal_vchi_buffer_init(dev->mmal_instance, &buf->mmal);
> + return 0;
> +}
> +
> +static int bcm2835_isp_buf_prepare(struct vb2_buffer *vb)
> +{
> + struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> + struct bcm2835_isp_buffer *buf =
> + container_of(vb2, struct bcm2835_isp_buffer, vb);
> + struct dma_buf *dma_buf;
> + int ret;
> +
> + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: type: %d ptr %p\n",
> + __func__, vb->vb2_queue->type, vb);
> +
> + if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> + if (vb2->field == V4L2_FIELD_ANY)
> + vb2->field = V4L2_FIELD_NONE;
> + if (vb2->field != V4L2_FIELD_NONE) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s field isn't supported\n", __func__);
> + return -EINVAL;
> + }
> + }
> +
> + if (vb2_plane_size(vb, 0) < node->q_data.sizeimage) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s data will not fit into plane (%lu < %lu)\n",
> + __func__, vb2_plane_size(vb, 0),
> + (long)node->q_data.sizeimage);
> + return -EINVAL;
> + }
> +
> + if (!V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> + vb2_set_plane_payload(vb, 0, node->q_data.sizeimage);
> +
> + switch (vb->memory) {
> + case VB2_MEMORY_DMABUF:
> + dma_buf = dma_buf_get(vb->planes[0].m.fd);
> +
> + if (dma_buf != buf->mmal.dma_buf) {
> + /*
> + * dmabuf either hasn't already been mapped, or it has
> + * changed.
> + */
> + if (buf->mmal.dma_buf) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s Buffer changed - why did the core not call cleanup?\n",
> + __func__);
> + bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
> + }
> +
> + buf->mmal.dma_buf = dma_buf;
> + } else {
> + /*
> + * Already have a reference to the buffer, so release it
> + * here.
> + */
> + dma_buf_put(dma_buf);
> + }
> + ret = 0;
> + break;
> + case VB2_MEMORY_MMAP:
> + /*
> + * We want to do this at init, but vb2_core_expbuf checks that
> + * the index < q->num_buffers, and q->num_buffers only gets
> + * updated once all the buffers are allocated.
> + */
> + if (!buf->mmal.dma_buf) {
> + ret = vb2_core_expbuf_dmabuf(vb->vb2_queue,
> + vb->vb2_queue->type,
> + vb->index, 0, O_CLOEXEC,
> + &buf->mmal.dma_buf);
> + v4l2_dbg(3, debug, &dev->v4l2_dev,
> + "%s: exporting ptr %p to dmabuf %p\n",
> + __func__, vb, buf->mmal.dma_buf);
> + if (ret)
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Failed to expbuf idx %d, ret %d\n",
> + __func__, vb->index, ret);
> + } else {
> + ret = 0;
> + }
> + break;
> + default:
> + ret = -EINVAL;
> + break;
> + }
> +
> + return ret;
> +}
> +
> +static void bcm2835_isp_node_buffer_queue(struct vb2_buffer *buf)
> +{
> + struct bcm2835_isp_node *node = vb2_get_drv_priv(buf->vb2_queue);
> + struct vb2_v4l2_buffer *vbuf =
> + container_of(buf, struct vb2_v4l2_buffer, vb2_buf);
> + struct bcm2835_isp_buffer *buffer =
> + container_of(vbuf, struct bcm2835_isp_buffer, vb);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> +
> + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: node %s[%d], buffer %p\n",
> + __func__, node->name, node->id, buffer);
> +
> + vb2_to_mmal_buffer(&buffer->mmal, &buffer->vb);
> + v4l2_dbg(3, debug, &dev->v4l2_dev,
> + "%s: node %s[%d] - submitting mmal dmabuf %p\n", __func__,
> + node->name, node->id, buffer->mmal.dma_buf);
> + vchiq_mmal_submit_buffer(dev->mmal_instance, get_port_data(node),
> + &buffer->mmal);
> +}
> +
> +static void bcm2835_isp_buffer_cleanup(struct vb2_buffer *vb)
> +{
> + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> + struct bcm2835_isp_buffer *buffer =
> + container_of(vb2, struct bcm2835_isp_buffer, vb);
> +
> + bcm2835_isp_mmal_buf_cleanup(&buffer->mmal);
> +}
> +
> +static int bcm2835_isp_node_start_streaming(struct vb2_queue *q,
> + unsigned int count)
> +{
> + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct vchiq_mmal_port *port = get_port_data(node);
> + int ret;
> +
> + v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d] (count %u)\n",
> + __func__, node->name, node->id, count);
> +
> + ret = vchiq_mmal_component_enable(dev->mmal_instance, dev->component);
> + if (ret) {
> + v4l2_err(&dev->v4l2_dev, "%s: Failed enabling component, ret %d\n",
> + __func__, ret);
> + return -EIO;
> + }
> +
> + node->sequence = 0;
> + port->cb_ctx = node;
> + ret = vchiq_mmal_port_enable(dev->mmal_instance, port,
> + mmal_buffer_cb);
> + if (!ret)
> + atomic_inc(&dev->num_streaming);
> + else
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Failed enabling port, ret %d\n", __func__, ret);
> +
It's not obvious from this code what happens with outstanding buffers if there
is an error. They should be returned to vb2 with state VB2_BUF_STATE_QUEUED.
Does that happen?
> + return ret;
> +}
> +
> +static void bcm2835_isp_node_stop_streaming(struct vb2_queue *q)
> +{
> + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct vchiq_mmal_port *port = get_port_data(node);
> + unsigned int i;
> + int ret;
> +
> + v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d], mmal port %p\n",
> + __func__, node->name, node->id, port);
> +
> + init_completion(&dev->frame_cmplt);
> +
> + /* Disable MMAL port - this will flush buffers back */
> + ret = vchiq_mmal_port_disable(dev->mmal_instance, port);
> + if (ret)
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Failed disabling %s port, ret %d\n", __func__,
> + node_is_output(node) ? "i/p" : "o/p",
> + ret);
> +
> + while (atomic_read(&port->buffers_with_vpu)) {
> + v4l2_dbg(1, debug, &dev->v4l2_dev,
> + "%s: Waiting for buffers to be returned - %d outstanding\n",
> + __func__, atomic_read(&port->buffers_with_vpu));
> + ret = wait_for_completion_timeout(&dev->frame_cmplt, HZ);
> + if (ret <= 0) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Timeout waiting for buffers to be returned - %d outstanding\n",
> + __func__,
> + atomic_read(&port->buffers_with_vpu));
> + break;
> + }
> + }
> +
> + /* Release the VCSM handle here to release the associated dmabuf */
> + for (i = 0; i < q->num_buffers; i++) {
> + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(q->bufs[i]);
> + struct bcm2835_isp_buffer *buf =
> + container_of(vb2, struct bcm2835_isp_buffer, vb);
> + bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
> + }
> +
> + atomic_dec(&dev->num_streaming);
> + /* If all ports disabled, then disable the component */
> + if (atomic_read(&dev->num_streaming) == 0) {
> + ret = vchiq_mmal_component_disable(dev->mmal_instance,
> + dev->component);
> + if (ret) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Failed disabling component, ret %d\n",
> + __func__, ret);
> + }
> + }
> +
> + /*
> + * Simply wait for any vb2 buffers to finish. We could take steps to
> + * make them complete more quickly if we care, or even return them
> + * ourselves.
> + */
> + vb2_wait_for_all_buffers(&node->queue);
> +}
> +
> +static const struct vb2_ops bcm2835_isp_node_queue_ops = {
> + .queue_setup = bcm2835_isp_node_queue_setup,
> + .buf_init = bcm2835_isp_buf_init,
> + .buf_prepare = bcm2835_isp_buf_prepare,
> + .buf_queue = bcm2835_isp_node_buffer_queue,
> + .buf_cleanup = bcm2835_isp_buffer_cleanup,
> + .start_streaming = bcm2835_isp_node_start_streaming,
> + .stop_streaming = bcm2835_isp_node_stop_streaming,
> +};
> +
> +static const
> +struct bcm2835_isp_fmt *get_default_format(struct bcm2835_isp_node *node)
> +{
> + return node->supported_fmts.list[0];
> +}
> +
> +static inline unsigned int get_bytesperline(int width,
> + const struct bcm2835_isp_fmt *fmt)
> +{
> + return ALIGN((width * fmt->depth) >> 3, fmt->bytesperline_align);
> +}
> +
> +static inline unsigned int get_sizeimage(int bpl, int width, int height,
> + const struct bcm2835_isp_fmt *fmt)
> +{
> + return (bpl * height * fmt->size_multiplier_x2) >> 1;
> +}
> +
> +static int bcm2835_isp_s_ctrl(struct v4l2_ctrl *ctrl)
> +{
> + struct bcm2835_isp_dev *dev =
> + container_of(ctrl->handler, struct bcm2835_isp_dev, ctrl_handler);
> + struct bcm2835_isp_node *node = &dev->node[0];
> + int ret = 0;
> +
> + /*
> + * The ISP firmware driver will ensure these settings are applied on
> + * a frame boundary, so we are safe to write them as they come in.
> + *
> + * Note that the bcm2835_isp_* param structures are identical to the
> + * mmal-parameters.h definitions. This avoids the need for unnecessary
> + * field-by-field copying between structures.
> + */
> + switch (ctrl->id) {
> + case V4L2_CID_RED_BALANCE:
> + dev->r_gain = ctrl->val;
> + ret = set_wb_gains(node);
> + break;
> + case V4L2_CID_BLUE_BALANCE:
> + dev->b_gain = ctrl->val;
> + ret = set_wb_gains(node);
> + break;
> + case V4L2_CID_DIGITAL_GAIN:
> + ret = set_digital_gain(node, ctrl->val);
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_CC_MATRIX:
> + ret = set_isp_param(node, MMAL_PARAMETER_CUSTOM_CCM,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_custom_ccm));
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_LENS_SHADING:
> + ret = set_isp_param(node, MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_lens_shading));
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL:
> + ret = set_isp_param(node, MMAL_PARAMETER_BLACK_LEVEL,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_black_level));
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_GEQ:
> + ret = set_isp_param(node, MMAL_PARAMETER_GEQ,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_geq));
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_GAMMA:
> + ret = set_isp_param(node, MMAL_PARAMETER_GAMMA,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_gamma));
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_DENOISE:
> + ret = set_isp_param(node, MMAL_PARAMETER_DENOISE,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_denoise));
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_SHARPEN:
> + ret = set_isp_param(node, MMAL_PARAMETER_SHARPEN,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_sharpen));
> + break;
> + case V4L2_CID_USER_BCM2835_ISP_DPC:
> + ret = set_isp_param(node, MMAL_PARAMETER_DPC,
> + ctrl->p_new.p_u8,
> + sizeof(struct bcm2835_isp_dpc));
> + break;
> + default:
> + v4l2_info(&dev->v4l2_dev, "Unrecognised control\n");
> + ret = -EINVAL;
> + }
> +
> + if (ret) {
> + v4l2_err(&dev->v4l2_dev, "%s: Failed setting ctrl \"%s\" (%08x), err %d\n",
> + __func__, ctrl->name, ctrl->id, ret);
> + ret = -EIO;
> + }
> +
> + return ret;
> +}
> +
> +static const struct v4l2_ctrl_ops bcm2835_isp_ctrl_ops = {
> + .s_ctrl = bcm2835_isp_s_ctrl,
> +};
> +
> +static const struct v4l2_file_operations bcm2835_isp_fops = {
> + .owner = THIS_MODULE,
> + .open = v4l2_fh_open,
> + .release = vb2_fop_release,
> + .poll = vb2_fop_poll,
> + .unlocked_ioctl = video_ioctl2,
> + .mmap = vb2_fop_mmap
> +};
> +
> +static int populate_qdata_fmt(struct v4l2_format *f,
> + struct bcm2835_isp_node *node)
> +{
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct bcm2835_isp_q_data *q_data = &node->q_data;
> + struct vchiq_mmal_port *port;
> + int ret;
> +
> + if (!node_is_stats(node)) {
> + v4l2_dbg(1, debug, &dev->v4l2_dev,
> + "%s: Setting pix format for type %d, wxh: %ux%u, fmt: %08x, size %u\n",
> + __func__, f->type, f->fmt.pix.width, f->fmt.pix.height,
> + f->fmt.pix.pixelformat, f->fmt.pix.sizeimage);
> +
> + q_data->fmt = find_format(f, node);
> + q_data->width = f->fmt.pix.width;
> + q_data->height = f->fmt.pix.height;
> + q_data->height = f->fmt.pix.height;
> +
> + /* All parameters should have been set correctly by try_fmt */
> + q_data->bytesperline = f->fmt.pix.bytesperline;
> + q_data->sizeimage = f->fmt.pix.sizeimage;
> + } else {
> + v4l2_dbg(1, debug, &dev->v4l2_dev,
> + "%s: Setting meta format for fmt: %08x, size %u\n",
> + __func__, f->fmt.meta.dataformat,
> + f->fmt.meta.buffersize);
> +
> + q_data->fmt = find_format(f, node);
> + q_data->width = 0;
> + q_data->height = 0;
> + q_data->bytesperline = 0;
> + q_data->sizeimage = f->fmt.meta.buffersize;
> + }
> +
> + v4l2_dbg(1, debug, &dev->v4l2_dev,
> + "%s: Calculated bpl as %u, size %u\n", __func__,
> + q_data->bytesperline, q_data->sizeimage);
> +
> + /* If we have a component then setup the port as well */
> + port = get_port_data(node);
> + setup_mmal_port_format(node, port);
> + ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
> + if (ret) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Failed vchiq_mmal_port_set_format on port, ret %d\n",
> + __func__, ret);
> + ret = -EINVAL;
> + }
> +
> + if (q_data->sizeimage < port->minimum_buffer.size) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s: Current buffer size of %u < min buf size %u - driver mismatch to MMAL\n",
> + __func__,
> + q_data->sizeimage,
> + port->minimum_buffer.size);
> + }
> +
> + v4l2_dbg(1, debug, &dev->v4l2_dev,
> + "%s: Set format for type %d, wxh: %dx%d, fmt: %08x, size %u\n",
> + __func__, f->type, q_data->width, q_data->height,
> + q_data->fmt->fourcc, q_data->sizeimage);
> +
> + return ret;
> +}
> +
> +static int bcm2835_isp_node_querycap(struct file *file, void *priv,
> + struct v4l2_capability *cap)
> +{
> + strscpy(cap->driver, BCM2835_ISP_NAME, sizeof(cap->driver));
> + strscpy(cap->card, BCM2835_ISP_NAME, sizeof(cap->card));
> + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> + BCM2835_ISP_NAME);
> +
> + return 0;
> +}
> +
> +static int bcm2835_isp_node_g_fmt(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct bcm2835_isp_node *node = video_drvdata(file);
> +
> + if (f->type != node->queue.type)
> + return -EINVAL;
> +
> + if (node_is_stats(node)) {
> + f->fmt.meta.dataformat = V4L2_META_FMT_BCM2835_ISP_STATS;
> + f->fmt.meta.buffersize =
> + get_port_data(node)->minimum_buffer.size;
> + } else {
> + struct bcm2835_isp_q_data *q_data = &node->q_data;
> +
> + f->fmt.pix.width = q_data->width;
> + f->fmt.pix.height = q_data->height;
> + f->fmt.pix.field = V4L2_FIELD_NONE;
> + f->fmt.pix.pixelformat = q_data->fmt->fourcc;
> + f->fmt.pix.bytesperline = q_data->bytesperline;
> + f->fmt.pix.sizeimage = q_data->sizeimage;
> + f->fmt.pix.colorspace = q_data->fmt->colorspace;
> + }
> +
> + return 0;
> +}
> +
> +static int bcm2835_isp_node_enum_fmt(struct file *file, void *priv,
> + struct v4l2_fmtdesc *f)
> +{
> + struct bcm2835_isp_node *node = video_drvdata(file);
> + struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
> +
> + if (f->type != node->queue.type)
> + return -EINVAL;
> +
> + if (f->index < fmts->num_entries) {
> + /* Format found */
> + f->pixelformat = fmts->list[f->index]->fourcc;
> + f->flags = fmts->list[f->index]->flags;
> + return 0;
> + }
> +
> + return -EINVAL;
> +}
> +
> +static int bcm2835_isp_enum_framesizes(struct file *file, void *priv,
> + struct v4l2_frmsizeenum *fsize)
> +{
> + struct bcm2835_isp_node *node = video_drvdata(file);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + const struct bcm2835_isp_fmt *fmt;
> +
> + if (node_is_stats(node) || fsize->index)
> + return -EINVAL;
> +
> + fmt = find_format_by_fourcc(fsize->pixel_format, node);
> + if (!fmt) {
> + v4l2_err(&dev->v4l2_dev, "Invalid pixel code: %x\n",
> + fsize->pixel_format);
> + return -EINVAL;
> + }
> +
> + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> + fsize->stepwise.min_width = MIN_DIM;
> + fsize->stepwise.max_width = MAX_DIM;
> + fsize->stepwise.step_width = fmt->step_size;
> +
> + fsize->stepwise.min_height = MIN_DIM;
> + fsize->stepwise.max_height = MAX_DIM;
> + fsize->stepwise.step_height = fmt->step_size;
> +
> + return 0;
> +}
> +
> +static int bcm2835_isp_node_try_fmt(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct bcm2835_isp_node *node = video_drvdata(file);
> + const struct bcm2835_isp_fmt *fmt;
> +
> + if (f->type != node->queue.type)
> + return -EINVAL;
> +
> + fmt = find_format(f, node);
> + if (!fmt)
> + fmt = get_default_format(node);
> +
> + if (!node_is_stats(node)) {
> + f->fmt.pix.width = max(min(f->fmt.pix.width, MAX_DIM),
> + MIN_DIM);
> + f->fmt.pix.height = max(min(f->fmt.pix.height, MAX_DIM),
> + MIN_DIM);
> +
> + f->fmt.pix.pixelformat = fmt->fourcc;
> + f->fmt.pix.colorspace = fmt->colorspace;
> + f->fmt.pix.bytesperline = get_bytesperline(f->fmt.pix.width,
> + fmt);
> + f->fmt.pix.field = V4L2_FIELD_NONE;
> + f->fmt.pix.sizeimage =
> + get_sizeimage(f->fmt.pix.bytesperline, f->fmt.pix.width,
> + f->fmt.pix.height, fmt);
> + } else {
> + f->fmt.meta.dataformat = fmt->fourcc;
> + f->fmt.meta.buffersize =
> + get_port_data(node)->minimum_buffer.size;
> + }
> +
> + return 0;
> +}
> +
> +static int bcm2835_isp_node_s_fmt(struct file *file, void *priv,
> + struct v4l2_format *f)
> +{
> + struct bcm2835_isp_node *node = video_drvdata(file);
> + int ret;
> +
> + if (f->type != node->queue.type)
> + return -EINVAL;
> +
> + ret = bcm2835_isp_node_try_fmt(file, priv, f);
> + if (ret)
> + return ret;
> +
> + v4l2_dbg(1, debug, &node_get_dev(node)->v4l2_dev,
> + "%s: Set format for node %s[%d]\n",
> + __func__, node->name, node->id);
> +
> + return populate_qdata_fmt(f, node);
> +}
> +
> +static int bcm2835_isp_node_s_selection(struct file *file, void *fh,
> + struct v4l2_selection *s)
> +{
> + struct mmal_parameter_crop crop;
> + struct bcm2835_isp_node *node = video_drvdata(file);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct vchiq_mmal_port *port = get_port_data(node);
> +
> + /* This return value is required fro V4L2 compliance. */
> + if (node_is_stats(node))
> + return -ENOTTY;
> +
> + if (!s->r.width || !s->r.height)
> + return -EINVAL;
I'm missing a check for s->target.
> +
> + /* Adjust the crop window if goes outside the frame dimensions. */
> + s->r.left = min((unsigned int)max(s->r.left, 0),
> + node->q_data.width - MIN_DIM);
> + s->r.top = min((unsigned int)max(s->r.top, 0),
> + node->q_data.height - MIN_DIM);
> + s->r.width = max(min(s->r.width, node->q_data.width - s->r.left),
> + MIN_DIM);
> + s->r.height = max(min(s->r.height, node->q_data.height - s->r.top),
> + MIN_DIM);
> +
> + crop.rect.x = s->r.left;
> + crop.rect.y = s->r.top;
> + crop.rect.width = s->r.width;
> + crop.rect.height = s->r.height;
> +
> + return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> + MMAL_PARAMETER_CROP,
> + &crop, sizeof(crop));
> +}
> +
> +static int bcm2835_isp_node_g_selection(struct file *file, void *fh,
> + struct v4l2_selection *s)
> +{
> + struct bcm2835_isp_node *node = video_drvdata(file);
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct vchiq_mmal_port *port = get_port_data(node);
> + struct mmal_parameter_crop crop;
> + u32 crop_size = sizeof(crop);
> + int ret;
> +
> + /* This return value is required for V4L2 compliance. */
> + if (node_is_stats(node))
> + return -ENOTTY;
> +
> + /* We can only return out an input crop. */
> + if (s->target != V4L2_SEL_TGT_CROP)
> + return -EINVAL;
No support for _TGT_CROP_DEFAULT/BOUNDS? Those are usually supported
by drivers and are typically set to the width/height of the current
format.
I recommend adding support for these targets.
> +
> + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance, port,
> + MMAL_PARAMETER_CROP,
> + &crop, &crop_size);
> + if (!ret)
> + return -EINVAL;
> +
> + s->r.left = crop.rect.x;
> + s->r.top = crop.rect.y;
> + s->r.width = crop.rect.width;
> + s->r.height = crop.rect.height;
> +
> + return 0;
> +}
> +
> +static int bcm3285_isp_subscribe_event(struct v4l2_fh *fh,
> + const struct v4l2_event_subscription *s)
> +{
> + switch (s->type) {
> + /* Cannot change source parameters dynamically at runtime. */
> + case V4L2_EVENT_SOURCE_CHANGE:
> + return -EINVAL;
> + case V4L2_EVENT_CTRL:
> + return v4l2_ctrl_subscribe_event(fh, s);
> + default:
> + return v4l2_event_subscribe(fh, s, 4, NULL);
> + }
> +}
> +
> +static const struct v4l2_ioctl_ops bcm2835_isp_node_ioctl_ops = {
> + .vidioc_querycap = bcm2835_isp_node_querycap,
> + .vidioc_g_fmt_vid_cap = bcm2835_isp_node_g_fmt,
> + .vidioc_g_fmt_vid_out = bcm2835_isp_node_g_fmt,
> + .vidioc_g_fmt_meta_cap = bcm2835_isp_node_g_fmt,
> + .vidioc_s_fmt_vid_cap = bcm2835_isp_node_s_fmt,
> + .vidioc_s_fmt_vid_out = bcm2835_isp_node_s_fmt,
> + .vidioc_s_fmt_meta_cap = bcm2835_isp_node_s_fmt,
> + .vidioc_try_fmt_vid_cap = bcm2835_isp_node_try_fmt,
> + .vidioc_try_fmt_vid_out = bcm2835_isp_node_try_fmt,
> + .vidioc_try_fmt_meta_cap = bcm2835_isp_node_try_fmt,
> + .vidioc_s_selection = bcm2835_isp_node_s_selection,
> + .vidioc_g_selection = bcm2835_isp_node_g_selection,
> +
> + .vidioc_enum_fmt_vid_cap = bcm2835_isp_node_enum_fmt,
> + .vidioc_enum_fmt_vid_out = bcm2835_isp_node_enum_fmt,
> + .vidioc_enum_fmt_meta_cap = bcm2835_isp_node_enum_fmt,
> + .vidioc_enum_framesizes = bcm2835_isp_enum_framesizes,
> +
> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> + .vidioc_querybuf = vb2_ioctl_querybuf,
> + .vidioc_qbuf = vb2_ioctl_qbuf,
> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> + .vidioc_expbuf = vb2_ioctl_expbuf,
> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> +
> + .vidioc_streamon = vb2_ioctl_streamon,
> + .vidioc_streamoff = vb2_ioctl_streamoff,
> +
> + .vidioc_subscribe_event = bcm3285_isp_subscribe_event,
> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> +};
> +
> +/*
> + * Size of the array to provide to the VPU when asking for the list of supported
> + * formats.
> + *
> + * The ISP component currently advertises 33 input formats, so add a small
> + * overhead on that.
> + */
> +#define MAX_SUPPORTED_ENCODINGS 40
> +
> +/* Populate node->supported_fmts with the formats supported by those ports. */
> +static int bcm2835_isp_get_supported_fmts(struct bcm2835_isp_node *node)
> +{
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> + struct bcm2835_isp_fmt const **list;
> + unsigned int i, j, num_encodings;
> + u32 fourccs[MAX_SUPPORTED_ENCODINGS];
> + u32 param_size = sizeof(fourccs);
> + int ret;
> +
> + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance,
> + get_port_data(node),
> + MMAL_PARAMETER_SUPPORTED_ENCODINGS,
> + &fourccs, ¶m_size);
> +
> + if (ret) {
> + if (ret == MMAL_MSG_STATUS_ENOSPC) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s: port has more encoding than we provided space for. Some are dropped.\n",
> + __func__);
> + num_encodings = MAX_SUPPORTED_ENCODINGS;
> + } else {
> + v4l2_err(&dev->v4l2_dev, "%s: get_param ret %u.\n",
> + __func__, ret);
> + return -EINVAL;
> + }
> + } else {
> + num_encodings = param_size / sizeof(u32);
> + }
> +
> + /*
> + * Assume at this stage that all encodings will be supported in V4L2.
> + * Any that aren't supported will waste a very small amount of memory.
> + */
> + list = devm_kzalloc(dev->dev,
> + sizeof(struct bcm2835_isp_fmt *) * num_encodings,
> + GFP_KERNEL);
> + if (!list)
> + return -ENOMEM;
> + node->supported_fmts.list = list;
> +
> + for (i = 0, j = 0; i < num_encodings; i++) {
> + const struct bcm2835_isp_fmt *fmt = get_fmt(fourccs[i]);
> +
> + if (fmt) {
> + list[j] = fmt;
> + j++;
> + }
> + }
> + node->supported_fmts.num_entries = j;
> +
> + return 0;
> +}
> +
> +/*
> + * Register a device node /dev/video<N> to go along with one of the ISP's input
> + * or output nodes.
> + */
> +static int register_node(struct bcm2835_isp_dev *dev,
> + struct bcm2835_isp_node *node,
> + int index)
> +{
> + struct video_device *vfd;
> + struct vb2_queue *queue;
> + int ret;
> +
> + mutex_init(&node->lock);
> +
> + node->dev = dev;
> + vfd = &node->vfd;
> + queue = &node->queue;
> + queue->type = index_to_queue_type(index);
> + /*
> + * Setup the node type-specific params.
> + *
> + * Only the OUTPUT node can set controls and crop windows. However,
> + * we must allow the s/g_selection ioctl on the stats node as v4l2
> + * compliance expects it to return a -ENOTTY, and the framework
> + * does not handle it if the ioctl is disabled.
> + */
> + switch (queue->type) {
> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> + vfd->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
> + node->id = index;
> + node->vfl_dir = VFL_DIR_TX;
> + node->name = "output";
> + break;
> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> + vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
> + /* First Capture node starts at id 0, etc. */
> + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
> + node->vfl_dir = VFL_DIR_RX;
> + node->name = "capture";
> + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
> + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_SELECTION);
> + v4l2_disable_ioctl(&node->vfd, VIDIOC_G_SELECTION);
> + break;
> + case V4L2_BUF_TYPE_META_CAPTURE:
> + vfd->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
> + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
> + node->vfl_dir = VFL_DIR_RX;
> + node->name = "stats";
> + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
Why not disable S/G_SELECTION for meta capture here rather than test for it
in the op functions?
> + break;
> + }
> +
> + /* We use the selection API instead of the old crop API. */
> + v4l2_disable_ioctl(vfd, VIDIOC_CROPCAP);
> + v4l2_disable_ioctl(vfd, VIDIOC_G_CROP);
> + v4l2_disable_ioctl(vfd, VIDIOC_S_CROP);
No need for this: the core handles this and will disable these ioctls
automatically.
> +
> + ret = bcm2835_isp_get_supported_fmts(node);
> + if (ret)
> + return ret;
> +
> + /* Initialise the the video node. */
> + vfd->vfl_type = VFL_TYPE_VIDEO;
> + vfd->fops = &bcm2835_isp_fops,
> + vfd->ioctl_ops = &bcm2835_isp_node_ioctl_ops,
> + vfd->minor = -1,
> + vfd->release = video_device_release_empty,
> + vfd->queue = &node->queue;
> + vfd->lock = &node->lock;
> + vfd->v4l2_dev = &dev->v4l2_dev;
> + vfd->vfl_dir = node->vfl_dir;
> +
> + node->q_data.fmt = get_default_format(node);
> + node->q_data.width = DEFAULT_DIM;
> + node->q_data.height = DEFAULT_DIM;
> + node->q_data.bytesperline =
> + get_bytesperline(DEFAULT_DIM, node->q_data.fmt);
> + node->q_data.sizeimage = node_is_stats(node) ?
> + get_port_data(node)->recommended_buffer.size :
> + get_sizeimage(node->q_data.bytesperline,
> + node->q_data.width,
> + node->q_data.height,
> + node->q_data.fmt);
> +
> + queue->io_modes = VB2_MMAP | VB2_DMABUF;
> + queue->drv_priv = node;
> + queue->ops = &bcm2835_isp_node_queue_ops;
> + queue->mem_ops = &vb2_dma_contig_memops;
> + queue->buf_struct_size = sizeof(struct bcm2835_isp_buffer);
> + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> + queue->dev = dev->dev;
> + queue->lock = &node->queue_lock;
> +
> + ret = vb2_queue_init(queue);
> + if (ret < 0) {
> + v4l2_info(&dev->v4l2_dev, "vb2_queue_init failed\n");
> + return ret;
> + }
> + node->queue_init = true;
> +
> + /* Define the device names */
> + snprintf(vfd->name, sizeof(node->vfd.name), "%s-%s%d", BCM2835_ISP_NAME,
> + node->name, node->id);
> +
> + ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr + index);
> + if (ret) {
> + v4l2_err(&dev->v4l2_dev,
> + "Failed to register video %s[%d] device node\n",
> + node->name, node->id);
> + return ret;
> + }
Move registering the video device to the end of this function.
Otherwise the output video device would be created (and available for
userspace) before the controls are added.
> +
> + node->registered = true;
> + video_set_drvdata(vfd, node);
> +
> + /* Set some controls and defaults, but only on the VIDEO_OUTPUT node. */
> + if (node_is_output(node)) {
> + unsigned int i;
> +
> + /* Use this ctrl template to assign all out ISP custom ctrls. */
> + struct v4l2_ctrl_config ctrl_template = {
> + .ops = &bcm2835_isp_ctrl_ops,
> + .type = V4L2_CTRL_TYPE_U8,
> + .def = 0,
> + .min = 0x00,
> + .max = 0xff,
> + .step = 1,
> + };
> +
> + v4l2_ctrl_handler_init(&dev->ctrl_handler, 4);
> +
> + dev->r_gain = 1000;
> + dev->b_gain = 1000;
> +
> + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> + V4L2_CID_RED_BALANCE, 1, 0xffff, 1,
> + dev->r_gain);
> +
> + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> + V4L2_CID_BLUE_BALANCE, 1, 0xffff, 1,
> + dev->b_gain);
> +
> + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> + V4L2_CID_DIGITAL_GAIN, 1, 0xffff, 1, 1000);
> +
> + for (i = 0; i < ARRAY_SIZE(custom_ctrls); i++) {
> + ctrl_template.name = custom_ctrls[i].name;
> + ctrl_template.id = custom_ctrls[i].id;
> + ctrl_template.dims[0] = custom_ctrls[i].size;
> + ctrl_template.flags = custom_ctrls[i].flags;
> + v4l2_ctrl_new_custom(&dev->ctrl_handler,
> + &ctrl_template, NULL);
> + }
> +
> + node->vfd.ctrl_handler = &dev->ctrl_handler;
Missing error check.
> + }
> +
> + v4l2_info(&dev->v4l2_dev,
> + "Device node %s[%d] registered as /dev/video%d\n",
> + node->name, node->id, vfd->num);
> +
> + return 0;
> +}
> +
> +/* Unregister one of the /dev/video<N> nodes associated with the ISP. */
> +static void unregister_node(struct bcm2835_isp_node *node)
> +{
> + struct bcm2835_isp_dev *dev = node_get_dev(node);
> +
> + v4l2_info(&dev->v4l2_dev,
> + "Unregistering node %s[%d] device node /dev/video%d\n",
> + node->name, node->id, node->vfd.num);
> +
> + if (node->queue_init)
> + vb2_queue_release(&node->queue);
> +
> + if (node->registered) {
> + video_unregister_device(&node->vfd);
> + if (node_is_output(node))
> + v4l2_ctrl_handler_free(&dev->ctrl_handler);
> + }
> +
> + /*
> + * node->supported_fmts.list is free'd automatically
> + * as a managed resource.
> + */
> + node->supported_fmts.list = NULL;
> + node->supported_fmts.num_entries = 0;
> + node->vfd.ctrl_handler = NULL;
> + node->registered = false;
> + node->queue_init = false;
> +}
> +
> +static void media_controller_unregister(struct bcm2835_isp_dev *dev)
> +{
> + unsigned int i;
> +
> + v4l2_info(&dev->v4l2_dev, "Unregister from media controller\n");
> +
> + if (dev->media_device_registered) {
> + media_device_unregister(&dev->mdev);
> + media_device_cleanup(&dev->mdev);
> + dev->media_device_registered = false;
> + }
> +
> + kfree(dev->entity.name);
> + dev->entity.name = NULL;
> +
> + if (dev->media_entity_registered) {
> + media_device_unregister_entity(&dev->entity);
> + dev->media_entity_registered = false;
> + }
> +
> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> + struct bcm2835_isp_node *node = &dev->node[i];
> +
> + if (node->media_node_registered) {
> + media_remove_intf_links(node->intf_link->intf);
> + media_entity_remove_links(&dev->node[i].vfd.entity);
> + media_devnode_remove(node->intf_devnode);
> + media_device_unregister_entity(&node->vfd.entity);
> + kfree(node->vfd.entity.name);
> + }
> + node->media_node_registered = false;
> + }
> +
> + dev->v4l2_dev.mdev = NULL;
> +}
> +
> +static int media_controller_register_node(struct bcm2835_isp_dev *dev, int num)
> +{
> + struct bcm2835_isp_node *node = &dev->node[num];
> + struct media_entity *entity = &node->vfd.entity;
> + int output = node_is_output(node);
> + char *name;
> + int ret;
> +
> + v4l2_info(&dev->v4l2_dev,
> + "Register %s node %d with media controller\n",
> + output ? "output" : "capture", num);
> + entity->obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
> + entity->function = MEDIA_ENT_F_IO_V4L;
> + entity->info.dev.major = VIDEO_MAJOR;
> + entity->info.dev.minor = node->vfd.minor;
> + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
> + if (!name) {
> + ret = -ENOMEM;
> + goto error_no_mem;
> + }
> + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "%s0-%s%d",
> + BCM2835_ISP_NAME, output ? "output" : "capture", num);
> + entity->name = name;
> + node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> + ret = media_entity_pads_init(entity, 1, &node->pad);
> + if (ret)
> + goto error_pads_init;
> + ret = media_device_register_entity(&dev->mdev, entity);
> + if (ret)
> + goto error_register_entity;
> +
> + node->intf_devnode = media_devnode_create(&dev->mdev,
> + MEDIA_INTF_T_V4L_VIDEO, 0,
> + VIDEO_MAJOR, node->vfd.minor);
> + if (!node->intf_devnode) {
> + ret = -ENOMEM;
> + goto error_devnode_create;
> + }
> +
> + node->intf_link = media_create_intf_link(entity,
> + &node->intf_devnode->intf,
> + MEDIA_LNK_FL_IMMUTABLE |
> + MEDIA_LNK_FL_ENABLED);
> + if (!node->intf_link) {
> + ret = -ENOMEM;
> + goto error_create_intf_link;
> + }
> +
> + if (output)
> + ret = media_create_pad_link(entity, 0, &dev->entity, num,
> + MEDIA_LNK_FL_IMMUTABLE |
> + MEDIA_LNK_FL_ENABLED);
> + else
> + ret = media_create_pad_link(&dev->entity, num, entity, 0,
> + MEDIA_LNK_FL_IMMUTABLE |
> + MEDIA_LNK_FL_ENABLED);
> + if (ret)
> + goto error_create_pad_link;
> +
> + dev->node[num].media_node_registered = true;
> + return 0;
> +
> +error_create_pad_link:
> + media_remove_intf_links(&node->intf_devnode->intf);
> +error_create_intf_link:
> + media_devnode_remove(node->intf_devnode);
> +error_devnode_create:
> + media_device_unregister_entity(&node->vfd.entity);
> +error_register_entity:
> +error_pads_init:
> + kfree(entity->name);
> + entity->name = NULL;
> +error_no_mem:
> + if (ret)
> + v4l2_info(&dev->v4l2_dev, "Error registering node\n");
> +
> + return ret;
> +}
> +
> +static int media_controller_register(struct bcm2835_isp_dev *dev)
> +{
> + char *name;
> + unsigned int i;
> + int ret;
> +
> + v4l2_dbg(2, debug, &dev->v4l2_dev, "Registering with media controller\n");
> + dev->mdev.dev = dev->dev;
> + strscpy(dev->mdev.model, "bcm2835-isp",
> + sizeof(dev->mdev.model));
> + strscpy(dev->mdev.bus_info, "platform:bcm2835-isp",
> + sizeof(dev->mdev.bus_info));
> + media_device_init(&dev->mdev);
> + dev->v4l2_dev.mdev = &dev->mdev;
> +
> + v4l2_dbg(2, debug, &dev->v4l2_dev, "Register entity for nodes\n");
> +
> + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
> + if (!name) {
> + ret = -ENOMEM;
> + goto done;
> + }
> + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "bcm2835_isp0");
> + dev->entity.name = name;
> + dev->entity.obj_type = MEDIA_ENTITY_TYPE_BASE;
> + dev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> +
> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> + dev->pad[i].flags = node_is_output(&dev->node[i]) ?
> + MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> + }
> +
> + ret = media_entity_pads_init(&dev->entity, BCM2835_ISP_NUM_NODES,
> + dev->pad);
> + if (ret)
> + goto done;
> +
> + ret = media_device_register_entity(&dev->mdev, &dev->entity);
> + if (ret)
> + goto done;
> +
> + dev->media_entity_registered = true;
> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> + ret = media_controller_register_node(dev, i);
> + if (ret)
> + goto done;
> + }
> +
> + ret = media_device_register(&dev->mdev);
> + if (!ret)
> + dev->media_device_registered = true;
> +done:
> + return ret;
> +}
> +
> +static int bcm2835_isp_remove(struct platform_device *pdev)
> +{
> + struct bcm2835_isp_dev *dev = platform_get_drvdata(pdev);
> + unsigned int i;
> +
> + media_controller_unregister(dev);
> +
> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++)
> + unregister_node(&dev->node[i]);
> +
> + v4l2_device_unregister(&dev->v4l2_dev);
> +
> + if (dev->component)
> + vchiq_mmal_component_finalise(dev->mmal_instance,
> + dev->component);
> +
> + vchiq_mmal_finalise(dev->mmal_instance);
> +
> + return 0;
> +}
> +
> +static int bcm2835_isp_probe(struct platform_device *pdev)
> +{
> + struct bcm2835_isp_dev *dev;
> + unsigned int i;
> + int ret;
> +
> + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> + if (!dev)
> + return -ENOMEM;
> +
> + dev->dev = &pdev->dev;
> +
> + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> + if (ret)
> + return ret;
> +
> + ret = vchiq_mmal_init(&dev->mmal_instance);
> + if (ret) {
> + v4l2_device_unregister(&dev->v4l2_dev);
> + return ret;
> + }
> +
> + ret = vchiq_mmal_component_init(dev->mmal_instance, "ril.isp",
> + &dev->component);
> + if (ret) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s: failed to create ril.isp component\n", __func__);
> + goto error;
> + }
> +
> + if ((dev->component->inputs != BCM2835_ISP_NUM_OUTPUTS) ||
> + (dev->component->outputs != BCM2835_ISP_NUM_CAPTURES +
> + BCM2835_ISP_NUM_METADATA)) {
> + v4l2_err(&dev->v4l2_dev,
> + "%s: ril.isp returned %d i/p (%d expected), %d o/p (%d expected) ports\n",
> + __func__, dev->component->inputs,
> + BCM2835_ISP_NUM_OUTPUTS,
> + dev->component->outputs,
> + BCM2835_ISP_NUM_CAPTURES + BCM2835_ISP_NUM_METADATA);
> + goto error;
> + }
> +
> + atomic_set(&dev->num_streaming, 0);
> +
> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> + struct bcm2835_isp_node *node = &dev->node[i];
> +
> + ret = register_node(dev, node, i);
> + if (ret)
> + goto error;
> + }
> +
> + ret = media_controller_register(dev);
> + if (ret)
> + goto error;
> +
> + platform_set_drvdata(pdev, dev);
> + v4l2_info(&dev->v4l2_dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
> + return 0;
> +
> +error:
> + bcm2835_isp_remove(pdev);
> +
> + return ret;
> +}
> +
> +static struct platform_driver bcm2835_isp_pdrv = {
> + .probe = bcm2835_isp_probe,
> + .remove = bcm2835_isp_remove,
> + .driver = {
> + .name = BCM2835_ISP_NAME,
> + },
> +};
> +
> +module_platform_driver(bcm2835_isp_pdrv);
> +
> +MODULE_DESCRIPTION("BCM2835 ISP driver");
> +MODULE_AUTHOR("Naushir Patuck <naush@raspberrypi.com>");
> +MODULE_LICENSE("GPL");
> +MODULE_VERSION("1.0");
> +MODULE_ALIAS("platform:bcm2835-isp");
> diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> new file mode 100644
> index 000000000000..cfbb1063aad1
> --- /dev/null
> +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> @@ -0,0 +1,67 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Broadcom BCM2835 ISP driver
> + *
> + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> + *
> + * Author: Naushir Patuck (naush@raspberrypi.com)
> + *
> + */
> +
> +#ifndef BCM2835_ISP_CTRLS
> +#define BCM2835_ISP_CTRLS
> +
> +#include <linux/bcm2835-isp.h>
> +
> +struct bcm2835_isp_custom_ctrl {
> + const char *name;
> + u32 id;
> + u32 size;
> + u32 flags;
> +};
> +
> +static const struct bcm2835_isp_custom_ctrl custom_ctrls[] = {
> + {
> + .name = "Colour Correction Matrix",
> + .id = V4L2_CID_USER_BCM2835_ISP_CC_MATRIX,
> + .size = sizeof(struct bcm2835_isp_custom_ccm),
> + .flags = 0
> + }, {
> + .name = "Lens Shading",
> + .id = V4L2_CID_USER_BCM2835_ISP_LENS_SHADING,
> + .size = sizeof(struct bcm2835_isp_lens_shading),
> + .flags = V4L2_CTRL_FLAG_EXECUTE_ON_WRITE
> + }, {
> + .name = "Black Level",
> + .id = V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL,
> + .size = sizeof(struct bcm2835_isp_black_level),
> + .flags = 0
> + }, {
> + .name = "Green Equalisation",
> + .id = V4L2_CID_USER_BCM2835_ISP_GEQ,
> + .size = sizeof(struct bcm2835_isp_geq),
> + .flags = 0
> + }, {
> + .name = "Gamma",
> + .id = V4L2_CID_USER_BCM2835_ISP_GAMMA,
> + .size = sizeof(struct bcm2835_isp_gamma),
> + .flags = 0
> + }, {
> + .name = "Sharpen",
> + .id = V4L2_CID_USER_BCM2835_ISP_SHARPEN,
> + .size = sizeof(struct bcm2835_isp_sharpen),
> + .flags = 0
> + }, {
> + .name = "Denoise",
> + .id = V4L2_CID_USER_BCM2835_ISP_DENOISE,
> + .size = sizeof(struct bcm2835_isp_denoise),
> + .flags = 0
> + }, {
> + .name = "Defective Pixel Correction",
> + .id = V4L2_CID_USER_BCM2835_ISP_DPC,
> + .size = sizeof(struct bcm2835_isp_dpc),
> + .flags = 0
> + }
> +};
> +
> +#endif
> diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> new file mode 100644
> index 000000000000..af3bde152bb2
> --- /dev/null
> +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> @@ -0,0 +1,301 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Broadcom BCM2835 ISP driver
> + *
> + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> + *
> + * Author: Naushir Patuck (naush@raspberrypi.com)
> + *
> + */
> +
> +#ifndef BCM2835_ISP_FMTS
> +#define BCM2835_ISP_FMTS
> +
> +#include <linux/videodev2.h>
> +#include "vchiq-mmal/mmal-encodings.h"
> +
> +struct bcm2835_isp_fmt {
> + u32 fourcc;
> + int depth;
> + int bytesperline_align;
> + u32 flags;
> + u32 mmal_fmt;
> + int size_multiplier_x2;
> + enum v4l2_colorspace colorspace;
> + unsigned int step_size;
> +};
> +
> +struct bcm2835_isp_fmt_list {
> + struct bcm2835_isp_fmt const **list;
> + unsigned int num_entries;
> +};
> +
> +static const struct bcm2835_isp_fmt supported_formats[] = {
> + {
> + /* YUV formats */
> + .fourcc = V4L2_PIX_FMT_YUV420,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_I420,
> + .size_multiplier_x2 = 3,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVU420,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_YV12,
> + .size_multiplier_x2 = 3,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_NV12,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_NV12,
> + .size_multiplier_x2 = 3,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_NV21,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_NV21,
> + .size_multiplier_x2 = 3,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YUYV,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_YUYV,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_UYVY,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_UYVY,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_YVYU,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_YVYU,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_VYUY,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_VYUY,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> + .step_size = 2,
> + }, {
> + /* RGB formats */
> + .fourcc = V4L2_PIX_FMT_RGB24,
> + .depth = 24,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_RGB24,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .step_size = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_RGB565,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_RGB16,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .step_size = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_BGR24,
> + .depth = 24,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BGR24,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .step_size = 1,
> + }, {
> + .fourcc = V4L2_PIX_FMT_ABGR32,
> + .depth = 32,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BGRA,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_SRGB,
> + .step_size = 1,
> + }, {
> + /* Bayer formats */
> + /* 8 bit */
> + .fourcc = V4L2_PIX_FMT_SRGGB8,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB8,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR8,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR8,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG8,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG8,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG8,
> + .depth = 8,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG8,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + /* 10 bit */
> + .fourcc = V4L2_PIX_FMT_SRGGB10P,
> + .depth = 10,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR10P,
> + .depth = 10,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG10P,
> + .depth = 10,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG10P,
> + .depth = 10,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + /* 12 bit */
> + .fourcc = V4L2_PIX_FMT_SRGGB12P,
> + .depth = 12,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR12P,
> + .depth = 12,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG12P,
> + .depth = 12,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG12P,
> + .depth = 12,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12P,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + /* 16 bit */
> + .fourcc = V4L2_PIX_FMT_SRGGB16,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB16,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SBGGR16,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR16,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGRBG16,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG16,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + .fourcc = V4L2_PIX_FMT_SGBRG16,
> + .depth = 16,
> + .bytesperline_align = 32,
> + .flags = 0,
> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG16,
> + .size_multiplier_x2 = 2,
> + .colorspace = V4L2_COLORSPACE_RAW,
> + .step_size = 2,
> + }, {
> + /* ISP statistics format */
> + .fourcc = V4L2_META_FMT_BCM2835_ISP_STATS,
> + .mmal_fmt = MMAL_ENCODING_BRCM_STATS,
> + /* The rest are not valid fields for stats. */
> + }
> +};
> +
> +#endif
> diff --git a/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> new file mode 100644
> index 000000000000..edc452fa8318
> --- /dev/null
> +++ b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> @@ -0,0 +1,333 @@
> +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> +/*
> + * bcm2835-isp.h
> + *
> + * BCM2835 ISP driver - user space header file.
> + *
> + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> + *
> + * Author: Naushir Patuck (naush@raspberrypi.com)
> + *
> + */
> +
> +#ifndef __BCM2835_ISP_H_
> +#define __BCM2835_ISP_H_
> +
> +#include <linux/v4l2-controls.h>
> +
> +/* TODO: move the control IDs definitions to v4l2-controls.h */
> +#define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10c0)
As the TODO says: move this to v4l2-controls.h. Currently the 0x10c0 offset
clashes with V4L2_CID_USER_ATMEL_ISC_BASE, so that certainly should be fixed.
> +
> +/* TODO: move the formats definitions to videodev2.h */
> +/* 12 Y/CbCr 4:2:0 128 pixel wide column */
> +#define V4L2_PIX_FMT_NV12_COL128 v4l2_fourcc('N', 'C', '1', '2')
> +/* Y/CbCr 4:2:0 10bpc, 3x10 packed as 4 bytes in a 128 bytes / 96 pixel wide column */
> +#define V4L2_PIX_FMT_NV12_10_COL128 v4l2_fourcc('N', 'C', '3', '0')
> +/* Sensor Ancillary metadata */
> +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S')
> +/* BCM2835 ISP image statistics output */
> +#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A')
> +
> +#define V4L2_CID_USER_BCM2835_ISP_CC_MATRIX \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0001)
> +#define V4L2_CID_USER_BCM2835_ISP_LENS_SHADING \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0002)
> +#define V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0003)
> +#define V4L2_CID_USER_BCM2835_ISP_GEQ \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0004)
> +#define V4L2_CID_USER_BCM2835_ISP_GAMMA \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0005)
> +#define V4L2_CID_USER_BCM2835_ISP_DENOISE \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0006)
> +#define V4L2_CID_USER_BCM2835_ISP_SHARPEN \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
> +#define V4L2_CID_USER_BCM2835_ISP_DPC \
> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
There is no documentation for these controls. Specifically, it doesn't
tell you which struct should be used.
> +
> +/*
> + * All structs below are directly mapped onto the equivalent structs in
> + * drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> + * for convenience.
> + */
> +
> +/**
> + * struct bcm2835_isp_rational - Rational value type.
> + *
> + * @num: Numerator.
> + * @den: Denominator.
> + */
> +struct bcm2835_isp_rational {
> + __s32 num;
> + __s32 den;
Wouldn't it make more sense if den is a __u32?
> +};
> +
> +/**
> + * struct bcm2835_isp_ccm - Colour correction matrix.
> + *
> + * @ccm: 3x3 correction matrix coefficients.
> + * @offsets: 1x3 correction offsets.
> + */
> +struct bcm2835_isp_ccm {
> + struct bcm2835_isp_rational ccm[3][3];
> + __s32 offsets[3];
> +};
> +
> +/**
> + * struct bcm2835_isp_custom_ccm - Custom CCM applied with the
> + * V4L2_CID_USER_BCM2835_ISP_CC_MATRIX ctrl.
> + *
> + * @enabled: Enable custom CCM.
> + * @ccm: Custom CCM coefficients and offsets.
> + */
> +struct bcm2835_isp_custom_ccm {
> + __u32 enabled;
> + struct bcm2835_isp_ccm ccm;
> +};
> +
> +/**
> + * enum bcm2835_isp_gain_format - format of the gains in the lens shading
> + * tables used with the
> + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING ctrl.
> + *
> + * @GAIN_FORMAT_U0P8_1: Gains are u0.8 format, starting at 1.0
> + * @GAIN_FORMAT_U1P7_0: Gains are u1.7 format, starting at 0.0
> + * @GAIN_FORMAT_U1P7_1: Gains are u1.7 format, starting at 1.0
> + * @GAIN_FORMAT_U2P6_0: Gains are u2.6 format, starting at 0.0
> + * @GAIN_FORMAT_U2P6_1: Gains are u2.6 format, starting at 1.0
> + * @GAIN_FORMAT_U3P5_0: Gains are u3.5 format, starting at 0.0
> + * @GAIN_FORMAT_U3P5_1: Gains are u3.5 format, starting at 1.0
> + * @GAIN_FORMAT_U4P10: Gains are u4.10 format, starting at 0.0
> + */
> +enum bcm2835_isp_gain_format {
> + GAIN_FORMAT_U0P8_1 = 0,
> + GAIN_FORMAT_U1P7_0 = 1,
> + GAIN_FORMAT_U1P7_1 = 2,
> + GAIN_FORMAT_U2P6_0 = 3,
> + GAIN_FORMAT_U2P6_1 = 4,
> + GAIN_FORMAT_U3P5_0 = 5,
> + GAIN_FORMAT_U3P5_1 = 6,
> + GAIN_FORMAT_U4P10 = 7,
> +};
> +
> +/**
> + * struct bcm2835_isp_lens_shading - Lens shading tables supplied with the
> + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING
> + * ctrl.
> + *
> + * @enabled: Enable lens shading.
> + * @grid_cell_size: Size of grid cells in samples (16, 32, 64, 128 or 256).
> + * @grid_width: Width of lens shading tables in grid cells.
> + * @grid_stride: Row to row distance (in grid cells) between grid cells
> + * in the same horizontal location.
> + * @grid_height: Height of lens shading tables in grid cells.
> + * @mem_handle_table: Memory handle to the tables.
What sort of handle is this? I.e. where does it come from?
> + * @ref_transform: Reference transform - unsupported, please pass zero.
> + * @corner_sampled: Whether the gains are sampled at the corner points
> + * of the grid cells or in the cell centres.
> + * @gain_format: Format of the gains (see enum &bcm2835_isp_gain_format).
> + */
> +struct bcm2835_isp_lens_shading {
> + __u32 enabled;
> + __u32 grid_cell_size;
> + __u32 grid_width;
> + __u32 grid_stride;
> + __u32 grid_height;
> + __u32 mem_handle_table;
> + __u32 ref_transform;
> + __u32 corner_sampled;
> + __u32 gain_format;
> +};
> +
> +/**
> + * struct bcm2835_isp_black_level - Sensor black level set with the
> + * V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL ctrl.
> + *
> + * @enabled: Enable black level.
> + * @black_level_r: Black level for red channel.
> + * @black_level_g: Black level for green channels.
> + * @black_level_b: Black level for blue channel.
> + */
> +struct bcm2835_isp_black_level {
> + __u32 enabled;
> + __u16 black_level_r;
> + __u16 black_level_g;
> + __u16 black_level_b;
> + __u8 pad_[2]; /* Unused */
I prefer 'padding' over 'pad_'.
> +};
> +
> +/**
> + * struct bcm2835_isp_geq - Green equalisation parameters set with the
> + * V4L2_CID_USER_BCM2835_ISP_GEQ ctrl.
> + *
> + * @enabled: Enable green equalisation.
> + * @offset: Fixed offset of the green equalisation threshold.
> + * @slope: Slope of the green equalisation threshold.
> + */
> +struct bcm2835_isp_geq {
> + __u32 enabled;
> + __u32 offset;
> + struct bcm2835_isp_rational slope;
> +};
> +
> +#define BCM2835_NUM_GAMMA_PTS 33
> +
> +/**
> + * struct bcm2835_isp_gamma - Gamma parameters set with the
> + * V4L2_CID_USER_BCM2835_ISP_GAMMA ctrl.
> + *
> + * @enabled: Enable gamma adjustment.
> + * @X: X values of the points defining the gamma curve.
> + * Values should be scaled to 16 bits.
> + * @Y: Y values of the points defining the gamma curve.
> + * Values should be scaled to 16 bits.
I assume 0 == black and 0xffff == white (or max luminance)?
And so typically x[0] == y[0] == 0 and x[32] == y[32] == 0xffff?
> + */
> +struct bcm2835_isp_gamma {
> + __u32 enabled;
> + __u16 x[BCM2835_NUM_GAMMA_PTS];
> + __u16 y[BCM2835_NUM_GAMMA_PTS];
> +};
> +
> +/**
> + * struct bcm2835_isp_denoise - Denoise parameters set with the
> + * V4L2_CID_USER_BCM2835_ISP_DENOISE ctrl.
> + *
> + * @enabled: Enable denoise.
> + * @constant: Fixed offset of the noise threshold.
> + * @slope: Slope of the noise threshold.
> + * @strength: Denoise strength between 0.0 (off) and 1.0 (maximum).
> + */
> +struct bcm2835_isp_denoise {
> + __u32 enabled;
> + __u32 constant;
> + struct bcm2835_isp_rational slope;
> + struct bcm2835_isp_rational strength;
> +};
> +
> +/**
> + * struct bcm2835_isp_sharpen - Sharpen parameters set with the
> + * V4L2_CID_USER_BCM2835_ISP_SHARPEN ctrl.
> + *
> + * @enabled: Enable sharpening.
> + * @threshold: Threshold at which to start sharpening pixels.
> + * @strength: Strength with which pixel sharpening increases.
> + * @limit: Limit to the amount of sharpening applied.
> + */
> +struct bcm2835_isp_sharpen {
> + __u32 enabled;
> + struct bcm2835_isp_rational threshold;
> + struct bcm2835_isp_rational strength;
> + struct bcm2835_isp_rational limit;
> +};
> +
> +/**
> + * enum bcm2835_isp_dpc_mode - defective pixel correction (DPC) strength.
> + *
> + * @DPC_MODE_OFF: No DPC.
> + * @DPC_MODE_NORMAL: Normal DPC.
> + * @DPC_MODE_STRONG: Strong DPC.
> + */
> +enum bcm2835_isp_dpc_mode {
> + DPC_MODE_OFF = 0,
> + DPC_MODE_NORMAL = 1,
> + DPC_MODE_STRONG = 2,
> +};
> +
> +/**
> + * struct bcm2835_isp_dpc - Defective pixel correction (DPC) parameters set
> + * with the V4L2_CID_USER_BCM2835_ISP_DPC ctrl.
> + *
> + * @enabled: Enable DPC.
> + * @strength: DPC strength (see enum &bcm2835_isp_dpc_mode).
Isn't DPC_MODE_OFF equal to just setting 'enabled' to false? If so,
wouldn't the 'strength' field be sufficient?
> + */
> +struct bcm2835_isp_dpc {
> + __u32 enabled;
> + __u32 strength;
> +};
> +
> +/*
> + * ISP statistics structures.
> + *
> + * The bcm2835_isp_stats structure is generated at the output of the
> + * statistics node. Note that this does not directly map onto the statistics
> + * output of the ISP HW. Instead, the MMAL firmware code maps the HW statistics
> + * to the bcm2835_isp_stats structure.
> + */
> +#define DEFAULT_AWB_REGIONS_X 16
> +#define DEFAULT_AWB_REGIONS_Y 12
> +
> +#define NUM_HISTOGRAMS 2
> +#define NUM_HISTOGRAM_BINS 128
> +#define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
> +#define FLOATING_REGIONS 16
> +#define AGC_REGIONS 16
> +#define FOCUS_REGIONS 12
> +
> +/**
> + * struct bcm2835_isp_stats_hist - Histogram statistics
> + *
> + * @r_hist: Red channel histogram.
> + * @g_hist: Combined green channel histogram.
> + * @b_hist: Blue channel histogram.
> + */
> +struct bcm2835_isp_stats_hist {
> + __u32 r_hist[NUM_HISTOGRAM_BINS];
> + __u32 g_hist[NUM_HISTOGRAM_BINS];
> + __u32 b_hist[NUM_HISTOGRAM_BINS];
> +};
> +
> +/**
> + * struct bcm2835_isp_stats_region - Region sums.
> + *
> + * @counted: The number of 2x2 bayer tiles accumulated.
> + * @notcounted: The number of 2x2 bayer tiles not accumulated.
> + * @r_sum: Total sum of counted pixels in the red channel for a region.
> + * @g_sum: Total sum of counted pixels in the green channel for a region.
> + * @b_sum: Total sum of counted pixels in the blue channel for a region.
> + */
> +struct bcm2835_isp_stats_region {
> + __u32 counted;
> + __u32 notcounted;
> + __u64 r_sum;
> + __u64 g_sum;
> + __u64 b_sum;
> +};
> +
> +/**
> + * struct bcm2835_isp_stats_focus - Focus statistics.
> + *
> + * @contrast_val: Focus measure - accumulated output of the focus filter.
> + * In the first dimension, index [0] counts pixels below a
> + * preset threshold, and index [1] counts pixels above the
> + * threshold. In the second dimension, index [0] uses the
> + * first predefined filter, and index [1] uses the second
> + * predefined filter.
> + * @contrast_val_num: The number of counted pixels in the above accumulation.
> + */
> +struct bcm2835_isp_stats_focus {
> + __u64 contrast_val[2][2];
> + __u32 contrast_val_num[2][2];
> +};
> +
> +/**
> + * struct bcm2835_isp_stats - ISP statistics.
> + *
> + * @version: Version of the bcm2835_isp_stats structure.
> + * @size: Size of the bcm2835_isp_stats structure.
> + * @hist: Histogram statistics for the entire image.
> + * @awb_stats: Statistics for the regions defined for AWB calculations.
> + * @floating_stats: Statistics for arbitrarily placed (floating) regions.
> + * @agc_stats: Statistics for the regions defined for AGC calculations.
> + * @focus_stats: Focus filter statistics for the focus regions.
> + */
> +struct bcm2835_isp_stats {
> + __u32 version;
> + __u32 size;
> + struct bcm2835_isp_stats_hist hist[NUM_HISTOGRAMS];
> + struct bcm2835_isp_stats_region awb_stats[AWB_REGIONS];
> + struct bcm2835_isp_stats_region floating_stats[FLOATING_REGIONS];
> + struct bcm2835_isp_stats_region agc_stats[AGC_REGIONS];
> + struct bcm2835_isp_stats_focus focus_stats[FOCUS_REGIONS];
> +};
> +
> +#endif /* __BCM2835_ISP_H_ */
> diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> index 106f71e709df..072f3c755a68 100644
> --- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> +++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> @@ -5,4 +5,5 @@ config BCM2835_VCHIQ_MMAL
> help
> Enables the MMAL API over VCHIQ interface as used for the
> majority of the multimedia services on VideoCore.
> - Defaults to Y when the Broadcomd BCM2835 camera host is selected.
> + Defaults to Y when the Broadcomd BCM2835 camera host or ISP are
> + selected.
> diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> index 44ba91aa6d47..8d904fcce388 100644
> --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> @@ -100,6 +100,10 @@
> */
> #define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
>
> +/** ISP image statistics format
> + */
> +#define MMAL_ENCODING_BRCM_STATS MMAL_FOURCC('S', 'T', 'A', 'T')
> +
> /* }@ */
>
> /** \name Pre-defined audio encodings */
> diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> index 1793103b18fd..b3552af5cf8f 100644
> --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> @@ -221,6 +221,62 @@ enum mmal_parameter_camera_type {
> MMAL_PARAMETER_SHUTTER_SPEED,
> /**< Takes a @ref MMAL_PARAMETER_AWB_GAINS_T */
> MMAL_PARAMETER_CUSTOM_AWB_GAINS,
> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_SETTINGS_T */
> + MMAL_PARAMETER_CAMERA_SETTINGS,
> + /**< Takes a @ref MMAL_PARAMETER_PRIVACY_INDICATOR_T */
> + MMAL_PARAMETER_PRIVACY_INDICATOR,
> + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
> + MMAL_PARAMETER_VIDEO_DENOISE,
> + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
> + MMAL_PARAMETER_STILLS_DENOISE,
> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_ANNOTATE_T */
> + MMAL_PARAMETER_ANNOTATE,
> + /**< Takes a @ref MMAL_PARAMETER_STEREOSCOPIC_MODE_T */
> + MMAL_PARAMETER_STEREOSCOPIC_MODE,
> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_INTERFACE_T */
> + MMAL_PARAMETER_CAMERA_INTERFACE,
> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_CLOCKING_MODE_T */
> + MMAL_PARAMETER_CAMERA_CLOCKING_MODE,
> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_CONFIG_T */
> + MMAL_PARAMETER_CAMERA_RX_CONFIG,
> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_TIMING_T */
> + MMAL_PARAMETER_CAMERA_RX_TIMING,
> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> + MMAL_PARAMETER_DPF_CONFIG,
> +
> + /* 0x50 */
> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> + MMAL_PARAMETER_JPEG_RESTART_INTERVAL,
> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> + MMAL_PARAMETER_CAMERA_ISP_BLOCK_OVERRIDE,
> + /**< Takes a @ref MMAL_PARAMETER_LENS_SHADING_T */
> + MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> + MMAL_PARAMETER_BLACK_LEVEL,
> + /**< Takes a @ref MMAL_PARAMETER_RESIZE_T */
> + MMAL_PARAMETER_RESIZE_PARAMS,
> + /**< Takes a @ref MMAL_PARAMETER_CROP_T */
> + MMAL_PARAMETER_CROP,
> + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
> + MMAL_PARAMETER_OUTPUT_SHIFT,
> + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
> + MMAL_PARAMETER_CCM_SHIFT,
> + /**< Takes a @ref MMAL_PARAMETER_CUSTOM_CCM_T */
> + MMAL_PARAMETER_CUSTOM_CCM,
> + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
> + MMAL_PARAMETER_ANALOG_GAIN,
> + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
> + MMAL_PARAMETER_DIGITAL_GAIN,
> + /**< Takes a @ref MMAL_PARAMETER_DENOISE_T */
> + MMAL_PARAMETER_DENOISE,
> + /**< Takes a @ref MMAL_PARAMETER_SHARPEN_T */
> + MMAL_PARAMETER_SHARPEN,
> + /**< Takes a @ref MMAL_PARAMETER_GEQ_T */
> + MMAL_PARAMETER_GEQ,
> + /**< Tales a @ref MMAP_PARAMETER_DPC_T */
> + MMAL_PARAMETER_DPC,
> + /**< Tales a @ref MMAP_PARAMETER_GAMMA_T */
> + MMAL_PARAMETER_GAMMA,
> };
>
> struct mmal_parameter_rational {
> @@ -779,7 +835,102 @@ struct mmal_parameter_camera_info {
> struct mmal_parameter_camera_info_camera
> cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
> struct mmal_parameter_camera_info_flash
> - flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
> + flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
> +};
> +
> +struct mmal_parameter_ccm {
> + struct mmal_parameter_rational ccm[3][3];
> + s32 offsets[3];
> +};
> +
> +struct mmal_parameter_custom_ccm {
> + u32 enabled; /**< Enable the custom CCM. */
> + struct mmal_parameter_ccm ccm; /**< CCM to be used. */
> +};
> +
> +struct mmal_parameter_lens_shading {
> + u32 enabled;
> + u32 grid_cell_size;
> + u32 grid_width;
> + u32 grid_stride;
> + u32 grid_height;
> + u32 mem_handle_table;
> + u32 ref_transform;
> +};
> +
> +enum mmal_parameter_ls_gain_format_type {
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U0P8_1 = 0,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_0 = 1,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_1 = 2,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_0 = 3,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_1 = 4,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_0 = 5,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_1 = 6,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U4P10 = 7,
> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_DUMMY = 0x7FFFFFFF
> +};
> +
> +struct mmal_parameter_lens_shading_v2 {
> + u32 enabled;
> + u32 grid_cell_size;
> + u32 grid_width;
> + u32 grid_stride;
> + u32 grid_height;
> + u32 mem_handle_table;
> + u32 ref_transform;
> + u32 corner_sampled;
> + enum mmal_parameter_ls_gain_format_type gain_format;
> +};
> +
> +struct mmal_parameter_black_level {
> + u32 enabled;
> + u16 black_level_r;
> + u16 black_level_g;
> + u16 black_level_b;
> + u8 pad_[2]; /* Unused */
> +};
> +
> +struct mmal_parameter_geq {
> + u32 enabled;
> + u32 offset;
> + struct mmal_parameter_rational slope;
> +};
> +
> +#define MMAL_NUM_GAMMA_PTS 33
> +struct mmal_parameter_gamma {
> + u32 enabled;
> + u16 x[MMAL_NUM_GAMMA_PTS];
> + u16 y[MMAL_NUM_GAMMA_PTS];
> +};
> +
> +struct mmal_parameter_denoise {
> + u32 enabled;
> + u32 constant;
> + struct mmal_parameter_rational slope;
> + struct mmal_parameter_rational strength;
> +};
> +
> +struct mmal_parameter_sharpen {
> + u32 enabled;
> + struct mmal_parameter_rational threshold;
> + struct mmal_parameter_rational strength;
> + struct mmal_parameter_rational limit;
> +};
> +
> +enum mmal_dpc_mode {
> + MMAL_DPC_MODE_OFF = 0,
> + MMAL_DPC_MODE_NORMAL = 1,
> + MMAL_DPC_MODE_STRONG = 2,
> + MMAL_DPC_MODE_MAX = 0x7FFFFFFF,
> +};
> +
> +struct mmal_parameter_dpc {
> + u32 enabled;
> + u32 strength;
> +};
> +
> +struct mmal_parameter_crop {
> + struct vchiq_mmal_rect rect;
> };
>
> #endif
>
Regards,
Hans
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-18 12:02 ` Hans Verkuil
@ 2020-05-18 14:36 ` Dave Stevenson
2020-05-18 15:07 ` Hans Verkuil
2020-05-19 12:47 ` Naushir Patuck
1 sibling, 1 reply; 111+ messages in thread
From: Dave Stevenson @ 2020-05-18 14:36 UTC (permalink / raw)
To: Hans Verkuil
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
Hi Hans.
Thanks for the review.
A few of these I need to leave for Naush to answer.
Ideas On Board have been contracted to drive this upstreaming process,
so many of these will be left to them to address and generate a v2.
On Mon, 18 May 2020 at 13:02, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 04/05/2020 11:26, Laurent Pinchart wrote:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
> > chips.
> >
> > The driver interfaces though the VideoCore unit using the VCHIQ MMAL
> > interface.
> >
> > ISP driver upported from from RaspberryPi BSP at revision:
> > 6c3505be6c3e ("staging: vc04_services: isp: Make all references to bcm2835_isp_fmt const")
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > [Adapt to staging by moving all modifications that in the BSP are scattered
> > in core components inside this directory]
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> > .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 41 +
> > drivers/staging/vc04_services/Kconfig | 2 +
> > drivers/staging/vc04_services/Makefile | 1 +
> > .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> > .../vc04_services/bcm2835-isp/Makefile | 10 +
> > .../bcm2835-isp/bcm2835-v4l2-isp.c | 1632 +++++++++++++++++
> > .../bcm2835-isp/bcm2835_isp_ctrls.h | 67 +
> > .../bcm2835-isp/bcm2835_isp_fmts.h | 301 +++
> > .../include/uapi/linux/bcm2835-isp.h | 333 ++++
> > .../staging/vc04_services/vchiq-mmal/Kconfig | 3 +-
> > .../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
> > .../vchiq-mmal/mmal-parameters.h | 153 +-
> > 12 files changed, 2559 insertions(+), 2 deletions(-)
> > create mode 100644 drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> > create mode 100644 drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> >
>
> <snip>
>
> > diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> > new file mode 100644
> > index 000000000000..a32faab4b8dc
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> > @@ -0,0 +1,1632 @@
>
> <snip>
>
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Broadcom BCM2835 ISP driver
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/platform_device.h>
> > +
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "vchiq-mmal/mmal-msg.h"
> > +#include "vchiq-mmal/mmal-parameters.h"
> > +#include "vchiq-mmal/mmal-vchiq.h"
> > +
> > +#include "bcm2835_isp_ctrls.h"
> > +#include "bcm2835_isp_fmts.h"
> > +
> > +static unsigned int debug;
> > +module_param(debug, uint, 0644);
> > +MODULE_PARM_DESC(debug, "activates debug info");
> > +
> > +static unsigned int video_nr = 13;
> > +module_param(video_nr, uint, 0644);
> > +MODULE_PARM_DESC(video_nr, "base video device number");
> > +
> > +#define BCM2835_ISP_NAME "bcm2835-isp"
> > +#define BCM2835_ISP_ENTITY_NAME_LEN 32
> > +
> > +#define BCM2835_ISP_NUM_OUTPUTS 1
> > +#define BCM2835_ISP_NUM_CAPTURES 2
> > +#define BCM2835_ISP_NUM_METADATA 1
> > +
> > +#define BCM2835_ISP_NUM_NODES \
> > + (BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES + \
> > + BCM2835_ISP_NUM_METADATA)
> > +
> > +/* Default frame dimension of 1280 pixels. */
> > +#define DEFAULT_DIM 1280U
> > +/*
> > + * Maximum frame dimension of 16384 pixels. Even though the ISP runs in tiles,
> > + * have a sensible limit so that we do not create an excessive number of tiles
> > + * to process.
> > + */
> > +#define MAX_DIM 16384U
> > +/*
> > + * Minimum frame dimension of 64 pixels. Anything lower, and the tiling
> > + * algorihtm may not be able to cope when applying filter context.
> > + */
> > +#define MIN_DIM 64U
> > +
> > +/* Per-queue, driver-specific private data */
> > +struct bcm2835_isp_q_data {
> > + /*
> > + * These parameters should be treated as gospel, with everything else
> > + * being determined from them.
> > + */
> > + unsigned int bytesperline;
> > + unsigned int width;
> > + unsigned int height;
> > + unsigned int sizeimage;
> > + const struct bcm2835_isp_fmt *fmt;
> > +};
> > +
> > +/*
> > + * Structure to describe a single node /dev/video<N> which represents a single
> > + * input or output queue to the ISP device.
> > + */
> > +struct bcm2835_isp_node {
> > + int vfl_dir;
> > + unsigned int id;
> > + const char *name;
> > + struct video_device vfd;
> > + struct media_pad pad;
> > + struct media_intf_devnode *intf_devnode;
> > + struct media_link *intf_link;
> > + struct mutex lock; /* top level device node lock */
> > + struct mutex queue_lock;
> > +
> > + struct vb2_queue queue;
> > + unsigned int sequence;
> > +
> > + /* The list of formats supported on the node. */
> > + struct bcm2835_isp_fmt_list supported_fmts;
> > +
> > + struct bcm2835_isp_q_data q_data;
> > +
> > + /* Parent device structure */
> > + struct bcm2835_isp_dev *dev;
> > +
> > + bool registered;
> > + bool media_node_registered;
> > + bool queue_init;
> > +};
> > +
> > +/*
> > + * Structure representing the entire ISP device, comprising several input and
> > + * output nodes /dev/video<N>.
> > + */
> > +struct bcm2835_isp_dev {
> > + struct v4l2_device v4l2_dev;
> > + struct device *dev;
> > + struct v4l2_ctrl_handler ctrl_handler;
> > + struct media_device mdev;
> > + struct media_entity entity;
> > + bool media_device_registered;
> > + bool media_entity_registered;
> > + struct vchiq_mmal_instance *mmal_instance;
> > + struct vchiq_mmal_component *component;
> > + struct completion frame_cmplt;
> > +
> > + struct bcm2835_isp_node node[BCM2835_ISP_NUM_NODES];
> > + struct media_pad pad[BCM2835_ISP_NUM_NODES];
> > + atomic_t num_streaming;
> > +
> > + /* Image pipeline controls. */
> > + int r_gain;
> > + int b_gain;
> > +};
> > +
> > +struct bcm2835_isp_buffer {
> > + struct vb2_v4l2_buffer vb;
> > + struct mmal_buffer mmal;
> > +};
> > +
> > +static
> > +inline struct bcm2835_isp_dev *node_get_dev(struct bcm2835_isp_node *node)
> > +{
> > + return node->dev;
> > +}
> > +
> > +static inline bool node_is_output(struct bcm2835_isp_node *node)
> > +{
> > + return node->queue.type == V4L2_BUF_TYPE_VIDEO_OUTPUT;
> > +}
> > +
> > +static inline bool node_is_capture(struct bcm2835_isp_node *node)
> > +{
> > + return node->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > +}
> > +
> > +static inline bool node_is_stats(struct bcm2835_isp_node *node)
> > +{
> > + return node->queue.type == V4L2_BUF_TYPE_META_CAPTURE;
> > +}
> > +
> > +static inline enum v4l2_buf_type index_to_queue_type(int index)
> > +{
> > + if (index < BCM2835_ISP_NUM_OUTPUTS)
> > + return V4L2_BUF_TYPE_VIDEO_OUTPUT;
> > + else if (index < BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES)
> > + return V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > + else
> > + return V4L2_BUF_TYPE_META_CAPTURE;
> > +}
> > +
> > +static struct vchiq_mmal_port *get_port_data(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + if (!dev->component)
> > + return NULL;
> > +
> > + switch (node->queue.type) {
> > + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> > + return &dev->component->input[node->id];
> > + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> > + case V4L2_BUF_TYPE_META_CAPTURE:
> > + return &dev->component->output[node->id];
> > + default:
> > + v4l2_err(&dev->v4l2_dev, "%s: Invalid queue type %u\n",
> > + __func__, node->queue.type);
> > + break;
> > + }
> > + return NULL;
> > +}
> > +
> > +static int set_isp_param(struct bcm2835_isp_node *node, u32 parameter,
> > + void *value, u32 value_size)
> > +{
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> > + parameter, value, value_size);
> > +}
> > +
> > +static int set_wb_gains(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct mmal_parameter_awbgains gains = {
> > + .r_gain = { dev->r_gain, 1000 },
> > + .b_gain = { dev->b_gain, 1000 }
> > + };
> > +
> > + return set_isp_param(node, MMAL_PARAMETER_CUSTOM_AWB_GAINS,
> > + &gains, sizeof(gains));
> > +}
> > +
> > +static int set_digital_gain(struct bcm2835_isp_node *node, uint32_t gain)
> > +{
> > + struct mmal_parameter_rational digital_gain = {
> > + .num = gain,
> > + .den = 1000
> > + };
> > +
> > + return set_isp_param(node, MMAL_PARAMETER_DIGITAL_GAIN,
> > + &digital_gain, sizeof(digital_gain));
> > +}
> > +
> > +static const struct bcm2835_isp_fmt *get_fmt(u32 mmal_fmt)
> > +{
> > + unsigned int i;
> > +
> > + for (i = 0; i < ARRAY_SIZE(supported_formats); i++) {
> > + if (supported_formats[i].mmal_fmt == mmal_fmt)
> > + return &supported_formats[i];
> > + }
> > + return NULL;
> > +}
> > +
> > +static const
> > +struct bcm2835_isp_fmt *find_format_by_fourcc(unsigned int fourcc,
> > + struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
> > + const struct bcm2835_isp_fmt *fmt;
> > + unsigned int i;
> > +
> > + for (i = 0; i < fmts->num_entries; i++) {
> > + fmt = fmts->list[i];
> > + if (fmt->fourcc == fourcc)
> > + return fmt;
> > + }
> > +
> > + return NULL;
> > +}
> > +
> > +static const
> > +struct bcm2835_isp_fmt *find_format(struct v4l2_format *f,
> > + struct bcm2835_isp_node *node)
> > +{
> > + return find_format_by_fourcc(node_is_stats(node) ?
> > + f->fmt.meta.dataformat :
> > + f->fmt.pix.pixelformat,
> > + node);
> > +}
> > +
> > +/* vb2_to_mmal_buffer() - converts vb2 buffer header to MMAL
> > + *
> > + * Copies all the required fields from a VB2 buffer to the MMAL buffer header,
> > + * ready for sending to the VPU.
> > + */
> > +static void vb2_to_mmal_buffer(struct mmal_buffer *buf,
> > + struct vb2_v4l2_buffer *vb2)
> > +{
> > + u64 pts;
> > +
> > + buf->mmal_flags = 0;
> > + if (vb2->flags & V4L2_BUF_FLAG_KEYFRAME)
> > + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_KEYFRAME;
> > +
> > + /* Data must be framed correctly as one frame per buffer. */
> > + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_FRAME_END;
> > +
> > + buf->length = vb2->vb2_buf.planes[0].bytesused;
> > + /*
> > + * Minor ambiguity in the V4L2 spec as to whether passing in a 0 length
> > + * buffer, or one with V4L2_BUF_FLAG_LAST set denotes end of stream.
> > + * Handle either.
> > + */
> > + if (!buf->length || vb2->flags & V4L2_BUF_FLAG_LAST)
> > + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_EOS;
> > +
> > + /* vb2 timestamps in nsecs, mmal in usecs */
> > + pts = vb2->vb2_buf.timestamp;
> > + do_div(pts, 1000);
> > + buf->pts = pts;
> > + buf->dts = MMAL_TIME_UNKNOWN;
> > +}
> > +
> > +static void mmal_buffer_cb(struct vchiq_mmal_instance *instance,
> > + struct vchiq_mmal_port *port, int status,
> > + struct mmal_buffer *mmal_buf)
> > +{
> > + struct bcm2835_isp_buffer *q_buf;
> > + struct bcm2835_isp_node *node = port->cb_ctx;
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vb2_v4l2_buffer *vb2;
> > +
> > + q_buf = container_of(mmal_buf, struct bcm2835_isp_buffer, mmal);
> > + vb2 = &q_buf->vb;
> > + v4l2_dbg(2, debug, &dev->v4l2_dev,
> > + "%s: port:%s[%d], status:%d, buf:%p, dmabuf:%p, length:%lu, flags %u, pts %lld\n",
> > + __func__, node_is_output(node) ? "input" : "output", node->id,
> > + status, mmal_buf, mmal_buf->dma_buf, mmal_buf->length,
> > + mmal_buf->mmal_flags, mmal_buf->pts);
> > +
> > + if (mmal_buf->cmd)
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Unexpected event on output callback - %08x\n",
> > + __func__, mmal_buf->cmd);
> > +
> > + if (status) {
> > + /* error in transfer */
> > + if (vb2) {
> > + /* there was a buffer with the error so return it */
> > + vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_ERROR);
> > + }
> > + return;
> > + }
> > +
> > + /* vb2 timestamps in nsecs, mmal in usecs */
> > + vb2->vb2_buf.timestamp = mmal_buf->pts * 1000;
> > + vb2->sequence = node->sequence++;
> > + vb2_set_plane_payload(&vb2->vb2_buf, 0, mmal_buf->length);
> > + vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_DONE);
> > +
> > + if (!port->enabled)
> > + complete(&dev->frame_cmplt);
> > +}
> > +
> > +static void setup_mmal_port_format(struct bcm2835_isp_node *node,
> > + struct vchiq_mmal_port *port)
> > +{
> > + struct bcm2835_isp_q_data *q_data = &node->q_data;
> > +
> > + port->format.encoding = q_data->fmt->mmal_fmt;
> > + /* Raw image format - set width/height */
> > + port->es.video.width = (q_data->bytesperline << 3) / q_data->fmt->depth;
> > + port->es.video.height = q_data->height;
> > + port->es.video.crop.width = q_data->width;
> > + port->es.video.crop.height = q_data->height;
> > + port->es.video.crop.x = 0;
> > + port->es.video.crop.y = 0;
> > +};
> > +
> > +static int setup_mmal_port(struct bcm2835_isp_node *node)
> > +{
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + unsigned int enable = 1;
> > + int ret;
> > +
> > + v4l2_dbg(2, debug, &dev->v4l2_dev, "%s: setup %s[%d]\n", __func__,
> > + node->name, node->id);
> > +
> > + vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> > + MMAL_PARAMETER_ZERO_COPY, &enable,
> > + sizeof(enable));
> > + setup_mmal_port_format(node, port);
> > + ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
> > + if (ret < 0) {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: vchiq_mmal_port_set_format failed\n",
> > + __func__);
> > + return ret;
> > + }
> > +
> > + if (node->q_data.sizeimage < port->minimum_buffer.size) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "buffer size mismatch sizeimage %u < min size %u\n",
> > + node->q_data.sizeimage, port->minimum_buffer.size);
> > + return -EINVAL;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
> > +{
> > + mmal_vchi_buffer_cleanup(mmal_buf);
> > +
> > + if (mmal_buf->dma_buf) {
> > + dma_buf_put(mmal_buf->dma_buf);
> > + mmal_buf->dma_buf = NULL;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_queue_setup(struct vb2_queue *q,
> > + unsigned int *nbuffers,
> > + unsigned int *nplanes,
> > + unsigned int sizes[],
> > + struct device *alloc_devs[])
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> > + struct vchiq_mmal_port *port;
> > + unsigned int size;
> > +
> > + if (setup_mmal_port(node))
> > + return -EINVAL;
> > +
> > + size = node->q_data.sizeimage;
> > + if (size == 0) {
> > + v4l2_info(&node_get_dev(node)->v4l2_dev,
> > + "%s: Image size unset in queue_setup for node %s[%d]\n",
> > + __func__, node->name, node->id);
> > + return -EINVAL;
> > + }
> > +
> > + if (*nplanes)
> > + return sizes[0] < size ? -EINVAL : 0;
> > +
> > + *nplanes = 1;
> > + sizes[0] = size;
> > +
> > + port = get_port_data(node);
> > + port->current_buffer.size = size;
> > +
> > + if (*nbuffers < port->minimum_buffer.num)
> > + *nbuffers = port->minimum_buffer.num;
> > +
> > + port->current_buffer.num = *nbuffers;
> > +
> > + v4l2_dbg(2, debug, &node_get_dev(node)->v4l2_dev,
> > + "%s: Image size %u, nbuffers %u for node %s[%d]\n",
> > + __func__, sizes[0], *nbuffers, node->name, node->id);
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_buf_init(struct vb2_buffer *vb)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> > + struct bcm2835_isp_buffer *buf =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > +
> > + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: vb %p\n", __func__, vb);
> > +
> > + buf->mmal.buffer = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
> > + buf->mmal.buffer_size = vb2_plane_size(&buf->vb.vb2_buf, 0);
> > + mmal_vchi_buffer_init(dev->mmal_instance, &buf->mmal);
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_buf_prepare(struct vb2_buffer *vb)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> > + struct bcm2835_isp_buffer *buf =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > + struct dma_buf *dma_buf;
> > + int ret;
> > +
> > + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: type: %d ptr %p\n",
> > + __func__, vb->vb2_queue->type, vb);
> > +
> > + if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> > + if (vb2->field == V4L2_FIELD_ANY)
> > + vb2->field = V4L2_FIELD_NONE;
> > + if (vb2->field != V4L2_FIELD_NONE) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s field isn't supported\n", __func__);
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (vb2_plane_size(vb, 0) < node->q_data.sizeimage) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s data will not fit into plane (%lu < %lu)\n",
> > + __func__, vb2_plane_size(vb, 0),
> > + (long)node->q_data.sizeimage);
> > + return -EINVAL;
> > + }
> > +
> > + if (!V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> > + vb2_set_plane_payload(vb, 0, node->q_data.sizeimage);
> > +
> > + switch (vb->memory) {
> > + case VB2_MEMORY_DMABUF:
> > + dma_buf = dma_buf_get(vb->planes[0].m.fd);
> > +
> > + if (dma_buf != buf->mmal.dma_buf) {
> > + /*
> > + * dmabuf either hasn't already been mapped, or it has
> > + * changed.
> > + */
> > + if (buf->mmal.dma_buf) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s Buffer changed - why did the core not call cleanup?\n",
> > + __func__);
> > + bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
> > + }
> > +
> > + buf->mmal.dma_buf = dma_buf;
> > + } else {
> > + /*
> > + * Already have a reference to the buffer, so release it
> > + * here.
> > + */
> > + dma_buf_put(dma_buf);
> > + }
> > + ret = 0;
> > + break;
> > + case VB2_MEMORY_MMAP:
> > + /*
> > + * We want to do this at init, but vb2_core_expbuf checks that
> > + * the index < q->num_buffers, and q->num_buffers only gets
> > + * updated once all the buffers are allocated.
> > + */
> > + if (!buf->mmal.dma_buf) {
> > + ret = vb2_core_expbuf_dmabuf(vb->vb2_queue,
> > + vb->vb2_queue->type,
> > + vb->index, 0, O_CLOEXEC,
> > + &buf->mmal.dma_buf);
> > + v4l2_dbg(3, debug, &dev->v4l2_dev,
> > + "%s: exporting ptr %p to dmabuf %p\n",
> > + __func__, vb, buf->mmal.dma_buf);
> > + if (ret)
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed to expbuf idx %d, ret %d\n",
> > + __func__, vb->index, ret);
> > + } else {
> > + ret = 0;
> > + }
> > + break;
> > + default:
> > + ret = -EINVAL;
> > + break;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static void bcm2835_isp_node_buffer_queue(struct vb2_buffer *buf)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(buf->vb2_queue);
> > + struct vb2_v4l2_buffer *vbuf =
> > + container_of(buf, struct vb2_v4l2_buffer, vb2_buf);
> > + struct bcm2835_isp_buffer *buffer =
> > + container_of(vbuf, struct bcm2835_isp_buffer, vb);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: node %s[%d], buffer %p\n",
> > + __func__, node->name, node->id, buffer);
> > +
> > + vb2_to_mmal_buffer(&buffer->mmal, &buffer->vb);
> > + v4l2_dbg(3, debug, &dev->v4l2_dev,
> > + "%s: node %s[%d] - submitting mmal dmabuf %p\n", __func__,
> > + node->name, node->id, buffer->mmal.dma_buf);
> > + vchiq_mmal_submit_buffer(dev->mmal_instance, get_port_data(node),
> > + &buffer->mmal);
> > +}
> > +
> > +static void bcm2835_isp_buffer_cleanup(struct vb2_buffer *vb)
> > +{
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> > + struct bcm2835_isp_buffer *buffer =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > +
> > + bcm2835_isp_mmal_buf_cleanup(&buffer->mmal);
> > +}
> > +
> > +static int bcm2835_isp_node_start_streaming(struct vb2_queue *q,
> > + unsigned int count)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + int ret;
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d] (count %u)\n",
> > + __func__, node->name, node->id, count);
> > +
> > + ret = vchiq_mmal_component_enable(dev->mmal_instance, dev->component);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev, "%s: Failed enabling component, ret %d\n",
> > + __func__, ret);
> > + return -EIO;
> > + }
> > +
> > + node->sequence = 0;
> > + port->cb_ctx = node;
> > + ret = vchiq_mmal_port_enable(dev->mmal_instance, port,
> > + mmal_buffer_cb);
> > + if (!ret)
> > + atomic_inc(&dev->num_streaming);
> > + else
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed enabling port, ret %d\n", __func__, ret);
> > +
>
> It's not obvious from this code what happens with outstanding buffers if there
> is an error. They should be returned to vb2 with state VB2_BUF_STATE_QUEUED.
> Does that happen?
No, it doesn't. That needs to be fixed.
> > + return ret;
> > +}
> > +
> > +static void bcm2835_isp_node_stop_streaming(struct vb2_queue *q)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + unsigned int i;
> > + int ret;
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d], mmal port %p\n",
> > + __func__, node->name, node->id, port);
> > +
> > + init_completion(&dev->frame_cmplt);
> > +
> > + /* Disable MMAL port - this will flush buffers back */
> > + ret = vchiq_mmal_port_disable(dev->mmal_instance, port);
> > + if (ret)
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed disabling %s port, ret %d\n", __func__,
> > + node_is_output(node) ? "i/p" : "o/p",
> > + ret);
> > +
> > + while (atomic_read(&port->buffers_with_vpu)) {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Waiting for buffers to be returned - %d outstanding\n",
> > + __func__, atomic_read(&port->buffers_with_vpu));
> > + ret = wait_for_completion_timeout(&dev->frame_cmplt, HZ);
> > + if (ret <= 0) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Timeout waiting for buffers to be returned - %d outstanding\n",
> > + __func__,
> > + atomic_read(&port->buffers_with_vpu));
> > + break;
> > + }
> > + }
> > +
> > + /* Release the VCSM handle here to release the associated dmabuf */
> > + for (i = 0; i < q->num_buffers; i++) {
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(q->bufs[i]);
> > + struct bcm2835_isp_buffer *buf =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > + bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
> > + }
> > +
> > + atomic_dec(&dev->num_streaming);
> > + /* If all ports disabled, then disable the component */
> > + if (atomic_read(&dev->num_streaming) == 0) {
> > + ret = vchiq_mmal_component_disable(dev->mmal_instance,
> > + dev->component);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed disabling component, ret %d\n",
> > + __func__, ret);
> > + }
> > + }
> > +
> > + /*
> > + * Simply wait for any vb2 buffers to finish. We could take steps to
> > + * make them complete more quickly if we care, or even return them
> > + * ourselves.
> > + */
> > + vb2_wait_for_all_buffers(&node->queue);
> > +}
> > +
> > +static const struct vb2_ops bcm2835_isp_node_queue_ops = {
> > + .queue_setup = bcm2835_isp_node_queue_setup,
> > + .buf_init = bcm2835_isp_buf_init,
> > + .buf_prepare = bcm2835_isp_buf_prepare,
> > + .buf_queue = bcm2835_isp_node_buffer_queue,
> > + .buf_cleanup = bcm2835_isp_buffer_cleanup,
> > + .start_streaming = bcm2835_isp_node_start_streaming,
> > + .stop_streaming = bcm2835_isp_node_stop_streaming,
> > +};
> > +
> > +static const
> > +struct bcm2835_isp_fmt *get_default_format(struct bcm2835_isp_node *node)
> > +{
> > + return node->supported_fmts.list[0];
> > +}
> > +
> > +static inline unsigned int get_bytesperline(int width,
> > + const struct bcm2835_isp_fmt *fmt)
> > +{
> > + return ALIGN((width * fmt->depth) >> 3, fmt->bytesperline_align);
> > +}
> > +
> > +static inline unsigned int get_sizeimage(int bpl, int width, int height,
> > + const struct bcm2835_isp_fmt *fmt)
> > +{
> > + return (bpl * height * fmt->size_multiplier_x2) >> 1;
> > +}
> > +
> > +static int bcm2835_isp_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > + struct bcm2835_isp_dev *dev =
> > + container_of(ctrl->handler, struct bcm2835_isp_dev, ctrl_handler);
> > + struct bcm2835_isp_node *node = &dev->node[0];
> > + int ret = 0;
> > +
> > + /*
> > + * The ISP firmware driver will ensure these settings are applied on
> > + * a frame boundary, so we are safe to write them as they come in.
> > + *
> > + * Note that the bcm2835_isp_* param structures are identical to the
> > + * mmal-parameters.h definitions. This avoids the need for unnecessary
> > + * field-by-field copying between structures.
> > + */
> > + switch (ctrl->id) {
> > + case V4L2_CID_RED_BALANCE:
> > + dev->r_gain = ctrl->val;
> > + ret = set_wb_gains(node);
> > + break;
> > + case V4L2_CID_BLUE_BALANCE:
> > + dev->b_gain = ctrl->val;
> > + ret = set_wb_gains(node);
> > + break;
> > + case V4L2_CID_DIGITAL_GAIN:
> > + ret = set_digital_gain(node, ctrl->val);
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_CC_MATRIX:
> > + ret = set_isp_param(node, MMAL_PARAMETER_CUSTOM_CCM,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_custom_ccm));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_LENS_SHADING:
> > + ret = set_isp_param(node, MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_lens_shading));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL:
> > + ret = set_isp_param(node, MMAL_PARAMETER_BLACK_LEVEL,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_black_level));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_GEQ:
> > + ret = set_isp_param(node, MMAL_PARAMETER_GEQ,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_geq));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_GAMMA:
> > + ret = set_isp_param(node, MMAL_PARAMETER_GAMMA,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_gamma));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_DENOISE:
> > + ret = set_isp_param(node, MMAL_PARAMETER_DENOISE,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_denoise));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_SHARPEN:
> > + ret = set_isp_param(node, MMAL_PARAMETER_SHARPEN,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_sharpen));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_DPC:
> > + ret = set_isp_param(node, MMAL_PARAMETER_DPC,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_dpc));
> > + break;
> > + default:
> > + v4l2_info(&dev->v4l2_dev, "Unrecognised control\n");
> > + ret = -EINVAL;
> > + }
> > +
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev, "%s: Failed setting ctrl \"%s\" (%08x), err %d\n",
> > + __func__, ctrl->name, ctrl->id, ret);
> > + ret = -EIO;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops bcm2835_isp_ctrl_ops = {
> > + .s_ctrl = bcm2835_isp_s_ctrl,
> > +};
> > +
> > +static const struct v4l2_file_operations bcm2835_isp_fops = {
> > + .owner = THIS_MODULE,
> > + .open = v4l2_fh_open,
> > + .release = vb2_fop_release,
> > + .poll = vb2_fop_poll,
> > + .unlocked_ioctl = video_ioctl2,
> > + .mmap = vb2_fop_mmap
> > +};
> > +
> > +static int populate_qdata_fmt(struct v4l2_format *f,
> > + struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct bcm2835_isp_q_data *q_data = &node->q_data;
> > + struct vchiq_mmal_port *port;
> > + int ret;
> > +
> > + if (!node_is_stats(node)) {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Setting pix format for type %d, wxh: %ux%u, fmt: %08x, size %u\n",
> > + __func__, f->type, f->fmt.pix.width, f->fmt.pix.height,
> > + f->fmt.pix.pixelformat, f->fmt.pix.sizeimage);
> > +
> > + q_data->fmt = find_format(f, node);
> > + q_data->width = f->fmt.pix.width;
> > + q_data->height = f->fmt.pix.height;
> > + q_data->height = f->fmt.pix.height;
> > +
> > + /* All parameters should have been set correctly by try_fmt */
> > + q_data->bytesperline = f->fmt.pix.bytesperline;
> > + q_data->sizeimage = f->fmt.pix.sizeimage;
> > + } else {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Setting meta format for fmt: %08x, size %u\n",
> > + __func__, f->fmt.meta.dataformat,
> > + f->fmt.meta.buffersize);
> > +
> > + q_data->fmt = find_format(f, node);
> > + q_data->width = 0;
> > + q_data->height = 0;
> > + q_data->bytesperline = 0;
> > + q_data->sizeimage = f->fmt.meta.buffersize;
> > + }
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Calculated bpl as %u, size %u\n", __func__,
> > + q_data->bytesperline, q_data->sizeimage);
> > +
> > + /* If we have a component then setup the port as well */
> > + port = get_port_data(node);
> > + setup_mmal_port_format(node, port);
> > + ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed vchiq_mmal_port_set_format on port, ret %d\n",
> > + __func__, ret);
> > + ret = -EINVAL;
> > + }
> > +
> > + if (q_data->sizeimage < port->minimum_buffer.size) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Current buffer size of %u < min buf size %u - driver mismatch to MMAL\n",
> > + __func__,
> > + q_data->sizeimage,
> > + port->minimum_buffer.size);
> > + }
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Set format for type %d, wxh: %dx%d, fmt: %08x, size %u\n",
> > + __func__, f->type, q_data->width, q_data->height,
> > + q_data->fmt->fourcc, q_data->sizeimage);
> > +
> > + return ret;
> > +}
> > +
> > +static int bcm2835_isp_node_querycap(struct file *file, void *priv,
> > + struct v4l2_capability *cap)
> > +{
> > + strscpy(cap->driver, BCM2835_ISP_NAME, sizeof(cap->driver));
> > + strscpy(cap->card, BCM2835_ISP_NAME, sizeof(cap->card));
> > + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > + BCM2835_ISP_NAME);
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_g_fmt(struct file *file, void *priv,
> > + struct v4l2_format *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + if (node_is_stats(node)) {
> > + f->fmt.meta.dataformat = V4L2_META_FMT_BCM2835_ISP_STATS;
> > + f->fmt.meta.buffersize =
> > + get_port_data(node)->minimum_buffer.size;
> > + } else {
> > + struct bcm2835_isp_q_data *q_data = &node->q_data;
> > +
> > + f->fmt.pix.width = q_data->width;
> > + f->fmt.pix.height = q_data->height;
> > + f->fmt.pix.field = V4L2_FIELD_NONE;
> > + f->fmt.pix.pixelformat = q_data->fmt->fourcc;
> > + f->fmt.pix.bytesperline = q_data->bytesperline;
> > + f->fmt.pix.sizeimage = q_data->sizeimage;
> > + f->fmt.pix.colorspace = q_data->fmt->colorspace;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_enum_fmt(struct file *file, void *priv,
> > + struct v4l2_fmtdesc *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + if (f->index < fmts->num_entries) {
> > + /* Format found */
> > + f->pixelformat = fmts->list[f->index]->fourcc;
> > + f->flags = fmts->list[f->index]->flags;
> > + return 0;
> > + }
> > +
> > + return -EINVAL;
> > +}
> > +
> > +static int bcm2835_isp_enum_framesizes(struct file *file, void *priv,
> > + struct v4l2_frmsizeenum *fsize)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + const struct bcm2835_isp_fmt *fmt;
> > +
> > + if (node_is_stats(node) || fsize->index)
> > + return -EINVAL;
> > +
> > + fmt = find_format_by_fourcc(fsize->pixel_format, node);
> > + if (!fmt) {
> > + v4l2_err(&dev->v4l2_dev, "Invalid pixel code: %x\n",
> > + fsize->pixel_format);
> > + return -EINVAL;
> > + }
> > +
> > + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > + fsize->stepwise.min_width = MIN_DIM;
> > + fsize->stepwise.max_width = MAX_DIM;
> > + fsize->stepwise.step_width = fmt->step_size;
> > +
> > + fsize->stepwise.min_height = MIN_DIM;
> > + fsize->stepwise.max_height = MAX_DIM;
> > + fsize->stepwise.step_height = fmt->step_size;
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_try_fmt(struct file *file, void *priv,
> > + struct v4l2_format *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + const struct bcm2835_isp_fmt *fmt;
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + fmt = find_format(f, node);
> > + if (!fmt)
> > + fmt = get_default_format(node);
> > +
> > + if (!node_is_stats(node)) {
> > + f->fmt.pix.width = max(min(f->fmt.pix.width, MAX_DIM),
> > + MIN_DIM);
> > + f->fmt.pix.height = max(min(f->fmt.pix.height, MAX_DIM),
> > + MIN_DIM);
> > +
> > + f->fmt.pix.pixelformat = fmt->fourcc;
> > + f->fmt.pix.colorspace = fmt->colorspace;
> > + f->fmt.pix.bytesperline = get_bytesperline(f->fmt.pix.width,
> > + fmt);
> > + f->fmt.pix.field = V4L2_FIELD_NONE;
> > + f->fmt.pix.sizeimage =
> > + get_sizeimage(f->fmt.pix.bytesperline, f->fmt.pix.width,
> > + f->fmt.pix.height, fmt);
> > + } else {
> > + f->fmt.meta.dataformat = fmt->fourcc;
> > + f->fmt.meta.buffersize =
> > + get_port_data(node)->minimum_buffer.size;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_s_fmt(struct file *file, void *priv,
> > + struct v4l2_format *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + int ret;
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + ret = bcm2835_isp_node_try_fmt(file, priv, f);
> > + if (ret)
> > + return ret;
> > +
> > + v4l2_dbg(1, debug, &node_get_dev(node)->v4l2_dev,
> > + "%s: Set format for node %s[%d]\n",
> > + __func__, node->name, node->id);
> > +
> > + return populate_qdata_fmt(f, node);
> > +}
> > +
> > +static int bcm2835_isp_node_s_selection(struct file *file, void *fh,
> > + struct v4l2_selection *s)
> > +{
> > + struct mmal_parameter_crop crop;
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > +
> > + /* This return value is required fro V4L2 compliance. */
> > + if (node_is_stats(node))
> > + return -ENOTTY;
> > +
> > + if (!s->r.width || !s->r.height)
> > + return -EINVAL;
>
> I'm missing a check for s->target.
Ack
> > +
> > + /* Adjust the crop window if goes outside the frame dimensions. */
> > + s->r.left = min((unsigned int)max(s->r.left, 0),
> > + node->q_data.width - MIN_DIM);
> > + s->r.top = min((unsigned int)max(s->r.top, 0),
> > + node->q_data.height - MIN_DIM);
> > + s->r.width = max(min(s->r.width, node->q_data.width - s->r.left),
> > + MIN_DIM);
> > + s->r.height = max(min(s->r.height, node->q_data.height - s->r.top),
> > + MIN_DIM);
> > +
> > + crop.rect.x = s->r.left;
> > + crop.rect.y = s->r.top;
> > + crop.rect.width = s->r.width;
> > + crop.rect.height = s->r.height;
> > +
> > + return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> > + MMAL_PARAMETER_CROP,
> > + &crop, sizeof(crop));
> > +}
> > +
> > +static int bcm2835_isp_node_g_selection(struct file *file, void *fh,
> > + struct v4l2_selection *s)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + struct mmal_parameter_crop crop;
> > + u32 crop_size = sizeof(crop);
> > + int ret;
> > +
> > + /* This return value is required for V4L2 compliance. */
> > + if (node_is_stats(node))
> > + return -ENOTTY;
> > +
> > + /* We can only return out an input crop. */
> > + if (s->target != V4L2_SEL_TGT_CROP)
> > + return -EINVAL;
>
> No support for _TGT_CROP_DEFAULT/BOUNDS? Those are usually supported
> by drivers and are typically set to the width/height of the current
> format.
>
> I recommend adding support for these targets.
Trying to find any good M2M drivers to use as an example was tricky,
but if the return value can be as simple as the current width/height,
then that's easy.
> > +
> > + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance, port,
> > + MMAL_PARAMETER_CROP,
> > + &crop, &crop_size);
> > + if (!ret)
> > + return -EINVAL;
> > +
> > + s->r.left = crop.rect.x;
> > + s->r.top = crop.rect.y;
> > + s->r.width = crop.rect.width;
> > + s->r.height = crop.rect.height;
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm3285_isp_subscribe_event(struct v4l2_fh *fh,
> > + const struct v4l2_event_subscription *s)
> > +{
> > + switch (s->type) {
> > + /* Cannot change source parameters dynamically at runtime. */
> > + case V4L2_EVENT_SOURCE_CHANGE:
> > + return -EINVAL;
> > + case V4L2_EVENT_CTRL:
> > + return v4l2_ctrl_subscribe_event(fh, s);
> > + default:
> > + return v4l2_event_subscribe(fh, s, 4, NULL);
> > + }
> > +}
> > +
> > +static const struct v4l2_ioctl_ops bcm2835_isp_node_ioctl_ops = {
> > + .vidioc_querycap = bcm2835_isp_node_querycap,
> > + .vidioc_g_fmt_vid_cap = bcm2835_isp_node_g_fmt,
> > + .vidioc_g_fmt_vid_out = bcm2835_isp_node_g_fmt,
> > + .vidioc_g_fmt_meta_cap = bcm2835_isp_node_g_fmt,
> > + .vidioc_s_fmt_vid_cap = bcm2835_isp_node_s_fmt,
> > + .vidioc_s_fmt_vid_out = bcm2835_isp_node_s_fmt,
> > + .vidioc_s_fmt_meta_cap = bcm2835_isp_node_s_fmt,
> > + .vidioc_try_fmt_vid_cap = bcm2835_isp_node_try_fmt,
> > + .vidioc_try_fmt_vid_out = bcm2835_isp_node_try_fmt,
> > + .vidioc_try_fmt_meta_cap = bcm2835_isp_node_try_fmt,
> > + .vidioc_s_selection = bcm2835_isp_node_s_selection,
> > + .vidioc_g_selection = bcm2835_isp_node_g_selection,
> > +
> > + .vidioc_enum_fmt_vid_cap = bcm2835_isp_node_enum_fmt,
> > + .vidioc_enum_fmt_vid_out = bcm2835_isp_node_enum_fmt,
> > + .vidioc_enum_fmt_meta_cap = bcm2835_isp_node_enum_fmt,
> > + .vidioc_enum_framesizes = bcm2835_isp_enum_framesizes,
> > +
> > + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> > + .vidioc_querybuf = vb2_ioctl_querybuf,
> > + .vidioc_qbuf = vb2_ioctl_qbuf,
> > + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> > + .vidioc_expbuf = vb2_ioctl_expbuf,
> > + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> > + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +
> > + .vidioc_streamon = vb2_ioctl_streamon,
> > + .vidioc_streamoff = vb2_ioctl_streamoff,
> > +
> > + .vidioc_subscribe_event = bcm3285_isp_subscribe_event,
> > + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +/*
> > + * Size of the array to provide to the VPU when asking for the list of supported
> > + * formats.
> > + *
> > + * The ISP component currently advertises 33 input formats, so add a small
> > + * overhead on that.
> > + */
> > +#define MAX_SUPPORTED_ENCODINGS 40
> > +
> > +/* Populate node->supported_fmts with the formats supported by those ports. */
> > +static int bcm2835_isp_get_supported_fmts(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct bcm2835_isp_fmt const **list;
> > + unsigned int i, j, num_encodings;
> > + u32 fourccs[MAX_SUPPORTED_ENCODINGS];
> > + u32 param_size = sizeof(fourccs);
> > + int ret;
> > +
> > + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance,
> > + get_port_data(node),
> > + MMAL_PARAMETER_SUPPORTED_ENCODINGS,
> > + &fourccs, ¶m_size);
> > +
> > + if (ret) {
> > + if (ret == MMAL_MSG_STATUS_ENOSPC) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: port has more encoding than we provided space for. Some are dropped.\n",
> > + __func__);
> > + num_encodings = MAX_SUPPORTED_ENCODINGS;
> > + } else {
> > + v4l2_err(&dev->v4l2_dev, "%s: get_param ret %u.\n",
> > + __func__, ret);
> > + return -EINVAL;
> > + }
> > + } else {
> > + num_encodings = param_size / sizeof(u32);
> > + }
> > +
> > + /*
> > + * Assume at this stage that all encodings will be supported in V4L2.
> > + * Any that aren't supported will waste a very small amount of memory.
> > + */
> > + list = devm_kzalloc(dev->dev,
> > + sizeof(struct bcm2835_isp_fmt *) * num_encodings,
> > + GFP_KERNEL);
> > + if (!list)
> > + return -ENOMEM;
> > + node->supported_fmts.list = list;
> > +
> > + for (i = 0, j = 0; i < num_encodings; i++) {
> > + const struct bcm2835_isp_fmt *fmt = get_fmt(fourccs[i]);
> > +
> > + if (fmt) {
> > + list[j] = fmt;
> > + j++;
> > + }
> > + }
> > + node->supported_fmts.num_entries = j;
> > +
> > + return 0;
> > +}
> > +
> > +/*
> > + * Register a device node /dev/video<N> to go along with one of the ISP's input
> > + * or output nodes.
> > + */
> > +static int register_node(struct bcm2835_isp_dev *dev,
> > + struct bcm2835_isp_node *node,
> > + int index)
> > +{
> > + struct video_device *vfd;
> > + struct vb2_queue *queue;
> > + int ret;
> > +
> > + mutex_init(&node->lock);
> > +
> > + node->dev = dev;
> > + vfd = &node->vfd;
> > + queue = &node->queue;
> > + queue->type = index_to_queue_type(index);
> > + /*
> > + * Setup the node type-specific params.
> > + *
> > + * Only the OUTPUT node can set controls and crop windows. However,
> > + * we must allow the s/g_selection ioctl on the stats node as v4l2
> > + * compliance expects it to return a -ENOTTY, and the framework
> > + * does not handle it if the ioctl is disabled.
> > + */
> > + switch (queue->type) {
> > + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> > + vfd->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
> > + node->id = index;
> > + node->vfl_dir = VFL_DIR_TX;
> > + node->name = "output";
> > + break;
> > + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> > + vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
> > + /* First Capture node starts at id 0, etc. */
> > + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
> > + node->vfl_dir = VFL_DIR_RX;
> > + node->name = "capture";
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_SELECTION);
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_G_SELECTION);
> > + break;
> > + case V4L2_BUF_TYPE_META_CAPTURE:
> > + vfd->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
> > + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
> > + node->vfl_dir = VFL_DIR_RX;
> > + node->name = "stats";
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
>
> Why not disable S/G_SELECTION for meta capture here rather than test for it
> in the op functions?
I'd have to ask Naush. There were some very odd interactions with
v4l2-compliance that he was struggling with.
> > + break;
> > + }
> > +
> > + /* We use the selection API instead of the old crop API. */
> > + v4l2_disable_ioctl(vfd, VIDIOC_CROPCAP);
> > + v4l2_disable_ioctl(vfd, VIDIOC_G_CROP);
> > + v4l2_disable_ioctl(vfd, VIDIOC_S_CROP);
>
> No need for this: the core handles this and will disable these ioctls
> automatically.
The core will ENABLE these if g_selection and s_selection are defined,
and uses mapping functions to try and convert between the two APIs.
This may be down to missing the target check in s_selection, but again
I seem to recall v4l2-compliance was reporting failures on these
ioctls.
> > +
> > + ret = bcm2835_isp_get_supported_fmts(node);
> > + if (ret)
> > + return ret;
> > +
> > + /* Initialise the the video node. */
> > + vfd->vfl_type = VFL_TYPE_VIDEO;
> > + vfd->fops = &bcm2835_isp_fops,
> > + vfd->ioctl_ops = &bcm2835_isp_node_ioctl_ops,
> > + vfd->minor = -1,
> > + vfd->release = video_device_release_empty,
> > + vfd->queue = &node->queue;
> > + vfd->lock = &node->lock;
> > + vfd->v4l2_dev = &dev->v4l2_dev;
> > + vfd->vfl_dir = node->vfl_dir;
> > +
> > + node->q_data.fmt = get_default_format(node);
> > + node->q_data.width = DEFAULT_DIM;
> > + node->q_data.height = DEFAULT_DIM;
> > + node->q_data.bytesperline =
> > + get_bytesperline(DEFAULT_DIM, node->q_data.fmt);
> > + node->q_data.sizeimage = node_is_stats(node) ?
> > + get_port_data(node)->recommended_buffer.size :
> > + get_sizeimage(node->q_data.bytesperline,
> > + node->q_data.width,
> > + node->q_data.height,
> > + node->q_data.fmt);
> > +
> > + queue->io_modes = VB2_MMAP | VB2_DMABUF;
> > + queue->drv_priv = node;
> > + queue->ops = &bcm2835_isp_node_queue_ops;
> > + queue->mem_ops = &vb2_dma_contig_memops;
> > + queue->buf_struct_size = sizeof(struct bcm2835_isp_buffer);
> > + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > + queue->dev = dev->dev;
> > + queue->lock = &node->queue_lock;
> > +
> > + ret = vb2_queue_init(queue);
> > + if (ret < 0) {
> > + v4l2_info(&dev->v4l2_dev, "vb2_queue_init failed\n");
> > + return ret;
> > + }
> > + node->queue_init = true;
> > +
> > + /* Define the device names */
> > + snprintf(vfd->name, sizeof(node->vfd.name), "%s-%s%d", BCM2835_ISP_NAME,
> > + node->name, node->id);
> > +
> > + ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr + index);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "Failed to register video %s[%d] device node\n",
> > + node->name, node->id);
> > + return ret;
> > + }
>
> Move registering the video device to the end of this function.
> Otherwise the output video device would be created (and available for
> userspace) before the controls are added.
Ack
> > +
> > + node->registered = true;
> > + video_set_drvdata(vfd, node);
> > +
> > + /* Set some controls and defaults, but only on the VIDEO_OUTPUT node. */
> > + if (node_is_output(node)) {
> > + unsigned int i;
> > +
> > + /* Use this ctrl template to assign all out ISP custom ctrls. */
> > + struct v4l2_ctrl_config ctrl_template = {
> > + .ops = &bcm2835_isp_ctrl_ops,
> > + .type = V4L2_CTRL_TYPE_U8,
> > + .def = 0,
> > + .min = 0x00,
> > + .max = 0xff,
> > + .step = 1,
> > + };
> > +
> > + v4l2_ctrl_handler_init(&dev->ctrl_handler, 4);
> > +
> > + dev->r_gain = 1000;
> > + dev->b_gain = 1000;
> > +
> > + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> > + V4L2_CID_RED_BALANCE, 1, 0xffff, 1,
> > + dev->r_gain);
> > +
> > + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> > + V4L2_CID_BLUE_BALANCE, 1, 0xffff, 1,
> > + dev->b_gain);
> > +
> > + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> > + V4L2_CID_DIGITAL_GAIN, 1, 0xffff, 1, 1000);
> > +
> > + for (i = 0; i < ARRAY_SIZE(custom_ctrls); i++) {
> > + ctrl_template.name = custom_ctrls[i].name;
> > + ctrl_template.id = custom_ctrls[i].id;
> > + ctrl_template.dims[0] = custom_ctrls[i].size;
> > + ctrl_template.flags = custom_ctrls[i].flags;
> > + v4l2_ctrl_new_custom(&dev->ctrl_handler,
> > + &ctrl_template, NULL);
> > + }
> > +
> > + node->vfd.ctrl_handler = &dev->ctrl_handler;
>
> Missing error check.
Ack
> > + }
> > +
> > + v4l2_info(&dev->v4l2_dev,
> > + "Device node %s[%d] registered as /dev/video%d\n",
> > + node->name, node->id, vfd->num);
> > +
> > + return 0;
> > +}
> > +
> > +/* Unregister one of the /dev/video<N> nodes associated with the ISP. */
> > +static void unregister_node(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + v4l2_info(&dev->v4l2_dev,
> > + "Unregistering node %s[%d] device node /dev/video%d\n",
> > + node->name, node->id, node->vfd.num);
> > +
> > + if (node->queue_init)
> > + vb2_queue_release(&node->queue);
> > +
> > + if (node->registered) {
> > + video_unregister_device(&node->vfd);
> > + if (node_is_output(node))
> > + v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > + }
> > +
> > + /*
> > + * node->supported_fmts.list is free'd automatically
> > + * as a managed resource.
> > + */
> > + node->supported_fmts.list = NULL;
> > + node->supported_fmts.num_entries = 0;
> > + node->vfd.ctrl_handler = NULL;
> > + node->registered = false;
> > + node->queue_init = false;
> > +}
> > +
> > +static void media_controller_unregister(struct bcm2835_isp_dev *dev)
> > +{
> > + unsigned int i;
> > +
> > + v4l2_info(&dev->v4l2_dev, "Unregister from media controller\n");
> > +
> > + if (dev->media_device_registered) {
> > + media_device_unregister(&dev->mdev);
> > + media_device_cleanup(&dev->mdev);
> > + dev->media_device_registered = false;
> > + }
> > +
> > + kfree(dev->entity.name);
> > + dev->entity.name = NULL;
> > +
> > + if (dev->media_entity_registered) {
> > + media_device_unregister_entity(&dev->entity);
> > + dev->media_entity_registered = false;
> > + }
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + struct bcm2835_isp_node *node = &dev->node[i];
> > +
> > + if (node->media_node_registered) {
> > + media_remove_intf_links(node->intf_link->intf);
> > + media_entity_remove_links(&dev->node[i].vfd.entity);
> > + media_devnode_remove(node->intf_devnode);
> > + media_device_unregister_entity(&node->vfd.entity);
> > + kfree(node->vfd.entity.name);
> > + }
> > + node->media_node_registered = false;
> > + }
> > +
> > + dev->v4l2_dev.mdev = NULL;
> > +}
> > +
> > +static int media_controller_register_node(struct bcm2835_isp_dev *dev, int num)
> > +{
> > + struct bcm2835_isp_node *node = &dev->node[num];
> > + struct media_entity *entity = &node->vfd.entity;
> > + int output = node_is_output(node);
> > + char *name;
> > + int ret;
> > +
> > + v4l2_info(&dev->v4l2_dev,
> > + "Register %s node %d with media controller\n",
> > + output ? "output" : "capture", num);
> > + entity->obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
> > + entity->function = MEDIA_ENT_F_IO_V4L;
> > + entity->info.dev.major = VIDEO_MAJOR;
> > + entity->info.dev.minor = node->vfd.minor;
> > + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
> > + if (!name) {
> > + ret = -ENOMEM;
> > + goto error_no_mem;
> > + }
> > + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "%s0-%s%d",
> > + BCM2835_ISP_NAME, output ? "output" : "capture", num);
> > + entity->name = name;
> > + node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> > + ret = media_entity_pads_init(entity, 1, &node->pad);
> > + if (ret)
> > + goto error_pads_init;
> > + ret = media_device_register_entity(&dev->mdev, entity);
> > + if (ret)
> > + goto error_register_entity;
> > +
> > + node->intf_devnode = media_devnode_create(&dev->mdev,
> > + MEDIA_INTF_T_V4L_VIDEO, 0,
> > + VIDEO_MAJOR, node->vfd.minor);
> > + if (!node->intf_devnode) {
> > + ret = -ENOMEM;
> > + goto error_devnode_create;
> > + }
> > +
> > + node->intf_link = media_create_intf_link(entity,
> > + &node->intf_devnode->intf,
> > + MEDIA_LNK_FL_IMMUTABLE |
> > + MEDIA_LNK_FL_ENABLED);
> > + if (!node->intf_link) {
> > + ret = -ENOMEM;
> > + goto error_create_intf_link;
> > + }
> > +
> > + if (output)
> > + ret = media_create_pad_link(entity, 0, &dev->entity, num,
> > + MEDIA_LNK_FL_IMMUTABLE |
> > + MEDIA_LNK_FL_ENABLED);
> > + else
> > + ret = media_create_pad_link(&dev->entity, num, entity, 0,
> > + MEDIA_LNK_FL_IMMUTABLE |
> > + MEDIA_LNK_FL_ENABLED);
> > + if (ret)
> > + goto error_create_pad_link;
> > +
> > + dev->node[num].media_node_registered = true;
> > + return 0;
> > +
> > +error_create_pad_link:
> > + media_remove_intf_links(&node->intf_devnode->intf);
> > +error_create_intf_link:
> > + media_devnode_remove(node->intf_devnode);
> > +error_devnode_create:
> > + media_device_unregister_entity(&node->vfd.entity);
> > +error_register_entity:
> > +error_pads_init:
> > + kfree(entity->name);
> > + entity->name = NULL;
> > +error_no_mem:
> > + if (ret)
> > + v4l2_info(&dev->v4l2_dev, "Error registering node\n");
> > +
> > + return ret;
> > +}
> > +
> > +static int media_controller_register(struct bcm2835_isp_dev *dev)
> > +{
> > + char *name;
> > + unsigned int i;
> > + int ret;
> > +
> > + v4l2_dbg(2, debug, &dev->v4l2_dev, "Registering with media controller\n");
> > + dev->mdev.dev = dev->dev;
> > + strscpy(dev->mdev.model, "bcm2835-isp",
> > + sizeof(dev->mdev.model));
> > + strscpy(dev->mdev.bus_info, "platform:bcm2835-isp",
> > + sizeof(dev->mdev.bus_info));
> > + media_device_init(&dev->mdev);
> > + dev->v4l2_dev.mdev = &dev->mdev;
> > +
> > + v4l2_dbg(2, debug, &dev->v4l2_dev, "Register entity for nodes\n");
> > +
> > + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
> > + if (!name) {
> > + ret = -ENOMEM;
> > + goto done;
> > + }
> > + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "bcm2835_isp0");
> > + dev->entity.name = name;
> > + dev->entity.obj_type = MEDIA_ENTITY_TYPE_BASE;
> > + dev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + dev->pad[i].flags = node_is_output(&dev->node[i]) ?
> > + MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > + }
> > +
> > + ret = media_entity_pads_init(&dev->entity, BCM2835_ISP_NUM_NODES,
> > + dev->pad);
> > + if (ret)
> > + goto done;
> > +
> > + ret = media_device_register_entity(&dev->mdev, &dev->entity);
> > + if (ret)
> > + goto done;
> > +
> > + dev->media_entity_registered = true;
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + ret = media_controller_register_node(dev, i);
> > + if (ret)
> > + goto done;
> > + }
> > +
> > + ret = media_device_register(&dev->mdev);
> > + if (!ret)
> > + dev->media_device_registered = true;
> > +done:
> > + return ret;
> > +}
> > +
> > +static int bcm2835_isp_remove(struct platform_device *pdev)
> > +{
> > + struct bcm2835_isp_dev *dev = platform_get_drvdata(pdev);
> > + unsigned int i;
> > +
> > + media_controller_unregister(dev);
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++)
> > + unregister_node(&dev->node[i]);
> > +
> > + v4l2_device_unregister(&dev->v4l2_dev);
> > +
> > + if (dev->component)
> > + vchiq_mmal_component_finalise(dev->mmal_instance,
> > + dev->component);
> > +
> > + vchiq_mmal_finalise(dev->mmal_instance);
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_probe(struct platform_device *pdev)
> > +{
> > + struct bcm2835_isp_dev *dev;
> > + unsigned int i;
> > + int ret;
> > +
> > + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > + if (!dev)
> > + return -ENOMEM;
> > +
> > + dev->dev = &pdev->dev;
> > +
> > + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > + if (ret)
> > + return ret;
> > +
> > + ret = vchiq_mmal_init(&dev->mmal_instance);
> > + if (ret) {
> > + v4l2_device_unregister(&dev->v4l2_dev);
> > + return ret;
> > + }
> > +
> > + ret = vchiq_mmal_component_init(dev->mmal_instance, "ril.isp",
> > + &dev->component);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: failed to create ril.isp component\n", __func__);
> > + goto error;
> > + }
> > +
> > + if ((dev->component->inputs != BCM2835_ISP_NUM_OUTPUTS) ||
> > + (dev->component->outputs != BCM2835_ISP_NUM_CAPTURES +
> > + BCM2835_ISP_NUM_METADATA)) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: ril.isp returned %d i/p (%d expected), %d o/p (%d expected) ports\n",
> > + __func__, dev->component->inputs,
> > + BCM2835_ISP_NUM_OUTPUTS,
> > + dev->component->outputs,
> > + BCM2835_ISP_NUM_CAPTURES + BCM2835_ISP_NUM_METADATA);
> > + goto error;
> > + }
> > +
> > + atomic_set(&dev->num_streaming, 0);
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + struct bcm2835_isp_node *node = &dev->node[i];
> > +
> > + ret = register_node(dev, node, i);
> > + if (ret)
> > + goto error;
> > + }
> > +
> > + ret = media_controller_register(dev);
> > + if (ret)
> > + goto error;
> > +
> > + platform_set_drvdata(pdev, dev);
> > + v4l2_info(&dev->v4l2_dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
> > + return 0;
> > +
> > +error:
> > + bcm2835_isp_remove(pdev);
> > +
> > + return ret;
> > +}
> > +
> > +static struct platform_driver bcm2835_isp_pdrv = {
> > + .probe = bcm2835_isp_probe,
> > + .remove = bcm2835_isp_remove,
> > + .driver = {
> > + .name = BCM2835_ISP_NAME,
> > + },
> > +};
> > +
> > +module_platform_driver(bcm2835_isp_pdrv);
> > +
> > +MODULE_DESCRIPTION("BCM2835 ISP driver");
> > +MODULE_AUTHOR("Naushir Patuck <naush@raspberrypi.com>");
> > +MODULE_LICENSE("GPL");
> > +MODULE_VERSION("1.0");
> > +MODULE_ALIAS("platform:bcm2835-isp");
> > diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> > new file mode 100644
> > index 000000000000..cfbb1063aad1
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> > @@ -0,0 +1,67 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Broadcom BCM2835 ISP driver
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#ifndef BCM2835_ISP_CTRLS
> > +#define BCM2835_ISP_CTRLS
> > +
> > +#include <linux/bcm2835-isp.h>
> > +
> > +struct bcm2835_isp_custom_ctrl {
> > + const char *name;
> > + u32 id;
> > + u32 size;
> > + u32 flags;
> > +};
> > +
> > +static const struct bcm2835_isp_custom_ctrl custom_ctrls[] = {
> > + {
> > + .name = "Colour Correction Matrix",
> > + .id = V4L2_CID_USER_BCM2835_ISP_CC_MATRIX,
> > + .size = sizeof(struct bcm2835_isp_custom_ccm),
> > + .flags = 0
> > + }, {
> > + .name = "Lens Shading",
> > + .id = V4L2_CID_USER_BCM2835_ISP_LENS_SHADING,
> > + .size = sizeof(struct bcm2835_isp_lens_shading),
> > + .flags = V4L2_CTRL_FLAG_EXECUTE_ON_WRITE
> > + }, {
> > + .name = "Black Level",
> > + .id = V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL,
> > + .size = sizeof(struct bcm2835_isp_black_level),
> > + .flags = 0
> > + }, {
> > + .name = "Green Equalisation",
> > + .id = V4L2_CID_USER_BCM2835_ISP_GEQ,
> > + .size = sizeof(struct bcm2835_isp_geq),
> > + .flags = 0
> > + }, {
> > + .name = "Gamma",
> > + .id = V4L2_CID_USER_BCM2835_ISP_GAMMA,
> > + .size = sizeof(struct bcm2835_isp_gamma),
> > + .flags = 0
> > + }, {
> > + .name = "Sharpen",
> > + .id = V4L2_CID_USER_BCM2835_ISP_SHARPEN,
> > + .size = sizeof(struct bcm2835_isp_sharpen),
> > + .flags = 0
> > + }, {
> > + .name = "Denoise",
> > + .id = V4L2_CID_USER_BCM2835_ISP_DENOISE,
> > + .size = sizeof(struct bcm2835_isp_denoise),
> > + .flags = 0
> > + }, {
> > + .name = "Defective Pixel Correction",
> > + .id = V4L2_CID_USER_BCM2835_ISP_DPC,
> > + .size = sizeof(struct bcm2835_isp_dpc),
> > + .flags = 0
> > + }
> > +};
> > +
> > +#endif
> > diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> > new file mode 100644
> > index 000000000000..af3bde152bb2
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> > @@ -0,0 +1,301 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Broadcom BCM2835 ISP driver
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#ifndef BCM2835_ISP_FMTS
> > +#define BCM2835_ISP_FMTS
> > +
> > +#include <linux/videodev2.h>
> > +#include "vchiq-mmal/mmal-encodings.h"
> > +
> > +struct bcm2835_isp_fmt {
> > + u32 fourcc;
> > + int depth;
> > + int bytesperline_align;
> > + u32 flags;
> > + u32 mmal_fmt;
> > + int size_multiplier_x2;
> > + enum v4l2_colorspace colorspace;
> > + unsigned int step_size;
> > +};
> > +
> > +struct bcm2835_isp_fmt_list {
> > + struct bcm2835_isp_fmt const **list;
> > + unsigned int num_entries;
> > +};
> > +
> > +static const struct bcm2835_isp_fmt supported_formats[] = {
> > + {
> > + /* YUV formats */
> > + .fourcc = V4L2_PIX_FMT_YUV420,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_I420,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YVU420,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_YV12,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_NV12,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_NV12,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_NV21,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_NV21,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YUYV,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_YUYV,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_UYVY,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_UYVY,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YVYU,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_YVYU,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_VYUY,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_VYUY,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + /* RGB formats */
> > + .fourcc = V4L2_PIX_FMT_RGB24,
> > + .depth = 24,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_RGB24,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_RGB565,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_RGB16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_BGR24,
> > + .depth = 24,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BGR24,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_ABGR32,
> > + .depth = 32,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BGRA,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + /* Bayer formats */
> > + /* 8 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* 10 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* 12 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* 16 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* ISP statistics format */
> > + .fourcc = V4L2_META_FMT_BCM2835_ISP_STATS,
> > + .mmal_fmt = MMAL_ENCODING_BRCM_STATS,
> > + /* The rest are not valid fields for stats. */
> > + }
> > +};
> > +
> > +#endif
> > diff --git a/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> > new file mode 100644
> > index 000000000000..edc452fa8318
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> > @@ -0,0 +1,333 @@
> > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> > +/*
> > + * bcm2835-isp.h
> > + *
> > + * BCM2835 ISP driver - user space header file.
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#ifndef __BCM2835_ISP_H_
> > +#define __BCM2835_ISP_H_
> > +
> > +#include <linux/v4l2-controls.h>
> > +
> > +/* TODO: move the control IDs definitions to v4l2-controls.h */
> > +#define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10c0)
>
> As the TODO says: move this to v4l2-controls.h. Currently the 0x10c0 offset
> clashes with V4L2_CID_USER_ATMEL_ISC_BASE, so that certainly should be fixed.
>
> > +
> > +/* TODO: move the formats definitions to videodev2.h */
> > +/* 12 Y/CbCr 4:2:0 128 pixel wide column */
> > +#define V4L2_PIX_FMT_NV12_COL128 v4l2_fourcc('N', 'C', '1', '2')
> > +/* Y/CbCr 4:2:0 10bpc, 3x10 packed as 4 bytes in a 128 bytes / 96 pixel wide column */
> > +#define V4L2_PIX_FMT_NV12_10_COL128 v4l2_fourcc('N', 'C', '3', '0')
These 2 seem to have been picked up into this patch set when they
really belong to the codec driver. I have got documentation written
for them.
> > +/* Sensor Ancillary metadata */
> > +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S')
> > +/* BCM2835 ISP image statistics output */
> > +#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A')
> > +
> > +#define V4L2_CID_USER_BCM2835_ISP_CC_MATRIX \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0001)
> > +#define V4L2_CID_USER_BCM2835_ISP_LENS_SHADING \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0002)
> > +#define V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0003)
> > +#define V4L2_CID_USER_BCM2835_ISP_GEQ \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0004)
> > +#define V4L2_CID_USER_BCM2835_ISP_GAMMA \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0005)
> > +#define V4L2_CID_USER_BCM2835_ISP_DENOISE \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0006)
> > +#define V4L2_CID_USER_BCM2835_ISP_SHARPEN \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
> > +#define V4L2_CID_USER_BCM2835_ISP_DPC \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
>
> There is no documentation for these controls. Specifically, it doesn't
> tell you which struct should be used.
>
> > +
> > +/*
> > + * All structs below are directly mapped onto the equivalent structs in
> > + * drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > + * for convenience.
> > + */
> > +
> > +/**
> > + * struct bcm2835_isp_rational - Rational value type.
> > + *
> > + * @num: Numerator.
> > + * @den: Denominator.
> > + */
> > +struct bcm2835_isp_rational {
> > + __s32 num;
> > + __s32 den;
>
> Wouldn't it make more sense if den is a __u32?
>
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_ccm - Colour correction matrix.
> > + *
> > + * @ccm: 3x3 correction matrix coefficients.
> > + * @offsets: 1x3 correction offsets.
> > + */
> > +struct bcm2835_isp_ccm {
> > + struct bcm2835_isp_rational ccm[3][3];
> > + __s32 offsets[3];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_custom_ccm - Custom CCM applied with the
> > + * V4L2_CID_USER_BCM2835_ISP_CC_MATRIX ctrl.
> > + *
> > + * @enabled: Enable custom CCM.
> > + * @ccm: Custom CCM coefficients and offsets.
> > + */
> > +struct bcm2835_isp_custom_ccm {
> > + __u32 enabled;
> > + struct bcm2835_isp_ccm ccm;
> > +};
> > +
> > +/**
> > + * enum bcm2835_isp_gain_format - format of the gains in the lens shading
> > + * tables used with the
> > + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING ctrl.
> > + *
> > + * @GAIN_FORMAT_U0P8_1: Gains are u0.8 format, starting at 1.0
> > + * @GAIN_FORMAT_U1P7_0: Gains are u1.7 format, starting at 0.0
> > + * @GAIN_FORMAT_U1P7_1: Gains are u1.7 format, starting at 1.0
> > + * @GAIN_FORMAT_U2P6_0: Gains are u2.6 format, starting at 0.0
> > + * @GAIN_FORMAT_U2P6_1: Gains are u2.6 format, starting at 1.0
> > + * @GAIN_FORMAT_U3P5_0: Gains are u3.5 format, starting at 0.0
> > + * @GAIN_FORMAT_U3P5_1: Gains are u3.5 format, starting at 1.0
> > + * @GAIN_FORMAT_U4P10: Gains are u4.10 format, starting at 0.0
> > + */
> > +enum bcm2835_isp_gain_format {
> > + GAIN_FORMAT_U0P8_1 = 0,
> > + GAIN_FORMAT_U1P7_0 = 1,
> > + GAIN_FORMAT_U1P7_1 = 2,
> > + GAIN_FORMAT_U2P6_0 = 3,
> > + GAIN_FORMAT_U2P6_1 = 4,
> > + GAIN_FORMAT_U3P5_0 = 5,
> > + GAIN_FORMAT_U3P5_1 = 6,
> > + GAIN_FORMAT_U4P10 = 7,
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_lens_shading - Lens shading tables supplied with the
> > + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING
> > + * ctrl.
> > + *
> > + * @enabled: Enable lens shading.
> > + * @grid_cell_size: Size of grid cells in samples (16, 32, 64, 128 or 256).
> > + * @grid_width: Width of lens shading tables in grid cells.
> > + * @grid_stride: Row to row distance (in grid cells) between grid cells
> > + * in the same horizontal location.
> > + * @grid_height: Height of lens shading tables in grid cells.
> > + * @mem_handle_table: Memory handle to the tables.
>
> What sort of handle is this? I.e. where does it come from?
This is being reworked to be a dmabuf fd allocated by userspace using dma-heaps.
> > + * @ref_transform: Reference transform - unsupported, please pass zero.
> > + * @corner_sampled: Whether the gains are sampled at the corner points
> > + * of the grid cells or in the cell centres.
> > + * @gain_format: Format of the gains (see enum &bcm2835_isp_gain_format).
> > + */
> > +struct bcm2835_isp_lens_shading {
> > + __u32 enabled;
> > + __u32 grid_cell_size;
> > + __u32 grid_width;
> > + __u32 grid_stride;
> > + __u32 grid_height;
> > + __u32 mem_handle_table;
> > + __u32 ref_transform;
> > + __u32 corner_sampled;
> > + __u32 gain_format;
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_black_level - Sensor black level set with the
> > + * V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL ctrl.
> > + *
> > + * @enabled: Enable black level.
> > + * @black_level_r: Black level for red channel.
> > + * @black_level_g: Black level for green channels.
> > + * @black_level_b: Black level for blue channel.
> > + */
> > +struct bcm2835_isp_black_level {
> > + __u32 enabled;
> > + __u16 black_level_r;
> > + __u16 black_level_g;
> > + __u16 black_level_b;
> > + __u8 pad_[2]; /* Unused */
>
> I prefer 'padding' over 'pad_'.
>
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_geq - Green equalisation parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_GEQ ctrl.
> > + *
> > + * @enabled: Enable green equalisation.
> > + * @offset: Fixed offset of the green equalisation threshold.
> > + * @slope: Slope of the green equalisation threshold.
> > + */
> > +struct bcm2835_isp_geq {
> > + __u32 enabled;
> > + __u32 offset;
> > + struct bcm2835_isp_rational slope;
> > +};
> > +
> > +#define BCM2835_NUM_GAMMA_PTS 33
> > +
> > +/**
> > + * struct bcm2835_isp_gamma - Gamma parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_GAMMA ctrl.
> > + *
> > + * @enabled: Enable gamma adjustment.
> > + * @X: X values of the points defining the gamma curve.
> > + * Values should be scaled to 16 bits.
> > + * @Y: Y values of the points defining the gamma curve.
> > + * Values should be scaled to 16 bits.
>
> I assume 0 == black and 0xffff == white (or max luminance)?
>
> And so typically x[0] == y[0] == 0 and x[32] == y[32] == 0xffff?
>
> > + */
> > +struct bcm2835_isp_gamma {
> > + __u32 enabled;
> > + __u16 x[BCM2835_NUM_GAMMA_PTS];
> > + __u16 y[BCM2835_NUM_GAMMA_PTS];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_denoise - Denoise parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_DENOISE ctrl.
> > + *
> > + * @enabled: Enable denoise.
> > + * @constant: Fixed offset of the noise threshold.
> > + * @slope: Slope of the noise threshold.
> > + * @strength: Denoise strength between 0.0 (off) and 1.0 (maximum).
> > + */
> > +struct bcm2835_isp_denoise {
> > + __u32 enabled;
> > + __u32 constant;
> > + struct bcm2835_isp_rational slope;
> > + struct bcm2835_isp_rational strength;
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_sharpen - Sharpen parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_SHARPEN ctrl.
> > + *
> > + * @enabled: Enable sharpening.
> > + * @threshold: Threshold at which to start sharpening pixels.
> > + * @strength: Strength with which pixel sharpening increases.
> > + * @limit: Limit to the amount of sharpening applied.
> > + */
> > +struct bcm2835_isp_sharpen {
> > + __u32 enabled;
> > + struct bcm2835_isp_rational threshold;
> > + struct bcm2835_isp_rational strength;
> > + struct bcm2835_isp_rational limit;
> > +};
> > +
> > +/**
> > + * enum bcm2835_isp_dpc_mode - defective pixel correction (DPC) strength.
> > + *
> > + * @DPC_MODE_OFF: No DPC.
> > + * @DPC_MODE_NORMAL: Normal DPC.
> > + * @DPC_MODE_STRONG: Strong DPC.
> > + */
> > +enum bcm2835_isp_dpc_mode {
> > + DPC_MODE_OFF = 0,
> > + DPC_MODE_NORMAL = 1,
> > + DPC_MODE_STRONG = 2,
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_dpc - Defective pixel correction (DPC) parameters set
> > + * with the V4L2_CID_USER_BCM2835_ISP_DPC ctrl.
> > + *
> > + * @enabled: Enable DPC.
> > + * @strength: DPC strength (see enum &bcm2835_isp_dpc_mode).
>
> Isn't DPC_MODE_OFF equal to just setting 'enabled' to false? If so,
> wouldn't the 'strength' field be sufficient?
>
> > + */
> > +struct bcm2835_isp_dpc {
> > + __u32 enabled;
> > + __u32 strength;
> > +};
> > +
> > +/*
> > + * ISP statistics structures.
> > + *
> > + * The bcm2835_isp_stats structure is generated at the output of the
> > + * statistics node. Note that this does not directly map onto the statistics
> > + * output of the ISP HW. Instead, the MMAL firmware code maps the HW statistics
> > + * to the bcm2835_isp_stats structure.
> > + */
> > +#define DEFAULT_AWB_REGIONS_X 16
> > +#define DEFAULT_AWB_REGIONS_Y 12
> > +
> > +#define NUM_HISTOGRAMS 2
> > +#define NUM_HISTOGRAM_BINS 128
> > +#define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
> > +#define FLOATING_REGIONS 16
> > +#define AGC_REGIONS 16
> > +#define FOCUS_REGIONS 12
> > +
> > +/**
> > + * struct bcm2835_isp_stats_hist - Histogram statistics
> > + *
> > + * @r_hist: Red channel histogram.
> > + * @g_hist: Combined green channel histogram.
> > + * @b_hist: Blue channel histogram.
> > + */
> > +struct bcm2835_isp_stats_hist {
> > + __u32 r_hist[NUM_HISTOGRAM_BINS];
> > + __u32 g_hist[NUM_HISTOGRAM_BINS];
> > + __u32 b_hist[NUM_HISTOGRAM_BINS];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_stats_region - Region sums.
> > + *
> > + * @counted: The number of 2x2 bayer tiles accumulated.
> > + * @notcounted: The number of 2x2 bayer tiles not accumulated.
> > + * @r_sum: Total sum of counted pixels in the red channel for a region.
> > + * @g_sum: Total sum of counted pixels in the green channel for a region.
> > + * @b_sum: Total sum of counted pixels in the blue channel for a region.
> > + */
> > +struct bcm2835_isp_stats_region {
> > + __u32 counted;
> > + __u32 notcounted;
> > + __u64 r_sum;
> > + __u64 g_sum;
> > + __u64 b_sum;
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_stats_focus - Focus statistics.
> > + *
> > + * @contrast_val: Focus measure - accumulated output of the focus filter.
> > + * In the first dimension, index [0] counts pixels below a
> > + * preset threshold, and index [1] counts pixels above the
> > + * threshold. In the second dimension, index [0] uses the
> > + * first predefined filter, and index [1] uses the second
> > + * predefined filter.
> > + * @contrast_val_num: The number of counted pixels in the above accumulation.
> > + */
> > +struct bcm2835_isp_stats_focus {
> > + __u64 contrast_val[2][2];
> > + __u32 contrast_val_num[2][2];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_stats - ISP statistics.
> > + *
> > + * @version: Version of the bcm2835_isp_stats structure.
> > + * @size: Size of the bcm2835_isp_stats structure.
> > + * @hist: Histogram statistics for the entire image.
> > + * @awb_stats: Statistics for the regions defined for AWB calculations.
> > + * @floating_stats: Statistics for arbitrarily placed (floating) regions.
> > + * @agc_stats: Statistics for the regions defined for AGC calculations.
> > + * @focus_stats: Focus filter statistics for the focus regions.
> > + */
> > +struct bcm2835_isp_stats {
> > + __u32 version;
> > + __u32 size;
> > + struct bcm2835_isp_stats_hist hist[NUM_HISTOGRAMS];
> > + struct bcm2835_isp_stats_region awb_stats[AWB_REGIONS];
> > + struct bcm2835_isp_stats_region floating_stats[FLOATING_REGIONS];
> > + struct bcm2835_isp_stats_region agc_stats[AGC_REGIONS];
> > + struct bcm2835_isp_stats_focus focus_stats[FOCUS_REGIONS];
> > +};
> > +
> > +#endif /* __BCM2835_ISP_H_ */
> > diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > index 106f71e709df..072f3c755a68 100644
> > --- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > +++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > @@ -5,4 +5,5 @@ config BCM2835_VCHIQ_MMAL
> > help
> > Enables the MMAL API over VCHIQ interface as used for the
> > majority of the multimedia services on VideoCore.
> > - Defaults to Y when the Broadcomd BCM2835 camera host is selected.
> > + Defaults to Y when the Broadcomd BCM2835 camera host or ISP are
> > + selected.
> > diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> > index 44ba91aa6d47..8d904fcce388 100644
> > --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> > +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> > @@ -100,6 +100,10 @@
> > */
> > #define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
> >
> > +/** ISP image statistics format
> > + */
> > +#define MMAL_ENCODING_BRCM_STATS MMAL_FOURCC('S', 'T', 'A', 'T')
> > +
> > /* }@ */
> >
> > /** \name Pre-defined audio encodings */
> > diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > index 1793103b18fd..b3552af5cf8f 100644
> > --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > @@ -221,6 +221,62 @@ enum mmal_parameter_camera_type {
> > MMAL_PARAMETER_SHUTTER_SPEED,
> > /**< Takes a @ref MMAL_PARAMETER_AWB_GAINS_T */
> > MMAL_PARAMETER_CUSTOM_AWB_GAINS,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_SETTINGS_T */
> > + MMAL_PARAMETER_CAMERA_SETTINGS,
> > + /**< Takes a @ref MMAL_PARAMETER_PRIVACY_INDICATOR_T */
> > + MMAL_PARAMETER_PRIVACY_INDICATOR,
> > + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
> > + MMAL_PARAMETER_VIDEO_DENOISE,
> > + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
> > + MMAL_PARAMETER_STILLS_DENOISE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_ANNOTATE_T */
> > + MMAL_PARAMETER_ANNOTATE,
> > + /**< Takes a @ref MMAL_PARAMETER_STEREOSCOPIC_MODE_T */
> > + MMAL_PARAMETER_STEREOSCOPIC_MODE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_INTERFACE_T */
> > + MMAL_PARAMETER_CAMERA_INTERFACE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_CLOCKING_MODE_T */
> > + MMAL_PARAMETER_CAMERA_CLOCKING_MODE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_CONFIG_T */
> > + MMAL_PARAMETER_CAMERA_RX_CONFIG,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_TIMING_T */
> > + MMAL_PARAMETER_CAMERA_RX_TIMING,
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_DPF_CONFIG,
> > +
> > + /* 0x50 */
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_JPEG_RESTART_INTERVAL,
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_CAMERA_ISP_BLOCK_OVERRIDE,
> > + /**< Takes a @ref MMAL_PARAMETER_LENS_SHADING_T */
> > + MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_BLACK_LEVEL,
> > + /**< Takes a @ref MMAL_PARAMETER_RESIZE_T */
> > + MMAL_PARAMETER_RESIZE_PARAMS,
> > + /**< Takes a @ref MMAL_PARAMETER_CROP_T */
> > + MMAL_PARAMETER_CROP,
> > + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
> > + MMAL_PARAMETER_OUTPUT_SHIFT,
> > + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
> > + MMAL_PARAMETER_CCM_SHIFT,
> > + /**< Takes a @ref MMAL_PARAMETER_CUSTOM_CCM_T */
> > + MMAL_PARAMETER_CUSTOM_CCM,
> > + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
> > + MMAL_PARAMETER_ANALOG_GAIN,
> > + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
> > + MMAL_PARAMETER_DIGITAL_GAIN,
> > + /**< Takes a @ref MMAL_PARAMETER_DENOISE_T */
> > + MMAL_PARAMETER_DENOISE,
> > + /**< Takes a @ref MMAL_PARAMETER_SHARPEN_T */
> > + MMAL_PARAMETER_SHARPEN,
> > + /**< Takes a @ref MMAL_PARAMETER_GEQ_T */
> > + MMAL_PARAMETER_GEQ,
> > + /**< Tales a @ref MMAP_PARAMETER_DPC_T */
> > + MMAL_PARAMETER_DPC,
> > + /**< Tales a @ref MMAP_PARAMETER_GAMMA_T */
> > + MMAL_PARAMETER_GAMMA,
> > };
> >
> > struct mmal_parameter_rational {
> > @@ -779,7 +835,102 @@ struct mmal_parameter_camera_info {
> > struct mmal_parameter_camera_info_camera
> > cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
> > struct mmal_parameter_camera_info_flash
> > - flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
> > + flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
> > +};
> > +
> > +struct mmal_parameter_ccm {
> > + struct mmal_parameter_rational ccm[3][3];
> > + s32 offsets[3];
> > +};
> > +
> > +struct mmal_parameter_custom_ccm {
> > + u32 enabled; /**< Enable the custom CCM. */
> > + struct mmal_parameter_ccm ccm; /**< CCM to be used. */
> > +};
> > +
> > +struct mmal_parameter_lens_shading {
> > + u32 enabled;
> > + u32 grid_cell_size;
> > + u32 grid_width;
> > + u32 grid_stride;
> > + u32 grid_height;
> > + u32 mem_handle_table;
> > + u32 ref_transform;
> > +};
> > +
> > +enum mmal_parameter_ls_gain_format_type {
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U0P8_1 = 0,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_0 = 1,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_1 = 2,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_0 = 3,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_1 = 4,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_0 = 5,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_1 = 6,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U4P10 = 7,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_DUMMY = 0x7FFFFFFF
> > +};
> > +
> > +struct mmal_parameter_lens_shading_v2 {
> > + u32 enabled;
> > + u32 grid_cell_size;
> > + u32 grid_width;
> > + u32 grid_stride;
> > + u32 grid_height;
> > + u32 mem_handle_table;
> > + u32 ref_transform;
> > + u32 corner_sampled;
> > + enum mmal_parameter_ls_gain_format_type gain_format;
> > +};
> > +
> > +struct mmal_parameter_black_level {
> > + u32 enabled;
> > + u16 black_level_r;
> > + u16 black_level_g;
> > + u16 black_level_b;
> > + u8 pad_[2]; /* Unused */
> > +};
> > +
> > +struct mmal_parameter_geq {
> > + u32 enabled;
> > + u32 offset;
> > + struct mmal_parameter_rational slope;
> > +};
> > +
> > +#define MMAL_NUM_GAMMA_PTS 33
> > +struct mmal_parameter_gamma {
> > + u32 enabled;
> > + u16 x[MMAL_NUM_GAMMA_PTS];
> > + u16 y[MMAL_NUM_GAMMA_PTS];
> > +};
> > +
> > +struct mmal_parameter_denoise {
> > + u32 enabled;
> > + u32 constant;
> > + struct mmal_parameter_rational slope;
> > + struct mmal_parameter_rational strength;
> > +};
> > +
> > +struct mmal_parameter_sharpen {
> > + u32 enabled;
> > + struct mmal_parameter_rational threshold;
> > + struct mmal_parameter_rational strength;
> > + struct mmal_parameter_rational limit;
> > +};
> > +
> > +enum mmal_dpc_mode {
> > + MMAL_DPC_MODE_OFF = 0,
> > + MMAL_DPC_MODE_NORMAL = 1,
> > + MMAL_DPC_MODE_STRONG = 2,
> > + MMAL_DPC_MODE_MAX = 0x7FFFFFFF,
> > +};
> > +
> > +struct mmal_parameter_dpc {
> > + u32 enabled;
> > + u32 strength;
> > +};
> > +
> > +struct mmal_parameter_crop {
> > + struct vchiq_mmal_rect rect;
> > };
> >
> > #endif
> >
Cheer
Dave
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-18 14:36 ` Dave Stevenson
@ 2020-05-18 15:07 ` Hans Verkuil
0 siblings, 0 replies; 111+ messages in thread
From: Hans Verkuil @ 2020-05-18 15:07 UTC (permalink / raw)
To: Dave Stevenson
Cc: Laurent Pinchart, Linux Media Mailing List, Kieran Bingham,
Jacopo Mondi, Niklas Söderlund, Naushir Patuck
On 18/05/2020 16:36, Dave Stevenson wrote:
> Hi Hans.
>
> Thanks for the review.
> A few of these I need to leave for Naush to answer.
> Ideas On Board have been contracted to drive this upstreaming process,
> so many of these will be left to them to address and generate a v2.
<snip>
>>> +static int bcm2835_isp_node_s_selection(struct file *file, void *fh,
>>> + struct v4l2_selection *s)
>>> +{
>>> + struct mmal_parameter_crop crop;
>>> + struct bcm2835_isp_node *node = video_drvdata(file);
>>> + struct bcm2835_isp_dev *dev = node_get_dev(node);
>>> + struct vchiq_mmal_port *port = get_port_data(node);
>>> +
>>> + /* This return value is required fro V4L2 compliance. */
>>> + if (node_is_stats(node))
>>> + return -ENOTTY;
>>> +
>>> + if (!s->r.width || !s->r.height)
>>> + return -EINVAL;
>>
>> I'm missing a check for s->target.
>
> Ack
>
>>> +
>>> + /* Adjust the crop window if goes outside the frame dimensions. */
>>> + s->r.left = min((unsigned int)max(s->r.left, 0),
>>> + node->q_data.width - MIN_DIM);
>>> + s->r.top = min((unsigned int)max(s->r.top, 0),
>>> + node->q_data.height - MIN_DIM);
>>> + s->r.width = max(min(s->r.width, node->q_data.width - s->r.left),
>>> + MIN_DIM);
>>> + s->r.height = max(min(s->r.height, node->q_data.height - s->r.top),
>>> + MIN_DIM);
>>> +
>>> + crop.rect.x = s->r.left;
>>> + crop.rect.y = s->r.top;
>>> + crop.rect.width = s->r.width;
>>> + crop.rect.height = s->r.height;
>>> +
>>> + return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
>>> + MMAL_PARAMETER_CROP,
>>> + &crop, sizeof(crop));
>>> +}
>>> +
>>> +static int bcm2835_isp_node_g_selection(struct file *file, void *fh,
>>> + struct v4l2_selection *s)
>>> +{
>>> + struct bcm2835_isp_node *node = video_drvdata(file);
>>> + struct bcm2835_isp_dev *dev = node_get_dev(node);
>>> + struct vchiq_mmal_port *port = get_port_data(node);
>>> + struct mmal_parameter_crop crop;
>>> + u32 crop_size = sizeof(crop);
>>> + int ret;
>>> +
>>> + /* This return value is required for V4L2 compliance. */
>>> + if (node_is_stats(node))
>>> + return -ENOTTY;
>>> +
>>> + /* We can only return out an input crop. */
>>> + if (s->target != V4L2_SEL_TGT_CROP)
>>> + return -EINVAL;
>>
>> No support for _TGT_CROP_DEFAULT/BOUNDS? Those are usually supported
>> by drivers and are typically set to the width/height of the current
>> format.
>>
>> I recommend adding support for these targets.
>
> Trying to find any good M2M drivers to use as an example was tricky,
> but if the return value can be as simple as the current width/height,
> then that's easy.
>
>>> +
>>> + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance, port,
>>> + MMAL_PARAMETER_CROP,
>>> + &crop, &crop_size);
>>> + if (!ret)
>>> + return -EINVAL;
>>> +
>>> + s->r.left = crop.rect.x;
>>> + s->r.top = crop.rect.y;
>>> + s->r.width = crop.rect.width;
>>> + s->r.height = crop.rect.height;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int bcm3285_isp_subscribe_event(struct v4l2_fh *fh,
>>> + const struct v4l2_event_subscription *s)
>>> +{
>>> + switch (s->type) {
>>> + /* Cannot change source parameters dynamically at runtime. */
>>> + case V4L2_EVENT_SOURCE_CHANGE:
>>> + return -EINVAL;
>>> + case V4L2_EVENT_CTRL:
>>> + return v4l2_ctrl_subscribe_event(fh, s);
>>> + default:
>>> + return v4l2_event_subscribe(fh, s, 4, NULL);
>>> + }
>>> +}
>>> +
>>> +static const struct v4l2_ioctl_ops bcm2835_isp_node_ioctl_ops = {
>>> + .vidioc_querycap = bcm2835_isp_node_querycap,
>>> + .vidioc_g_fmt_vid_cap = bcm2835_isp_node_g_fmt,
>>> + .vidioc_g_fmt_vid_out = bcm2835_isp_node_g_fmt,
>>> + .vidioc_g_fmt_meta_cap = bcm2835_isp_node_g_fmt,
>>> + .vidioc_s_fmt_vid_cap = bcm2835_isp_node_s_fmt,
>>> + .vidioc_s_fmt_vid_out = bcm2835_isp_node_s_fmt,
>>> + .vidioc_s_fmt_meta_cap = bcm2835_isp_node_s_fmt,
>>> + .vidioc_try_fmt_vid_cap = bcm2835_isp_node_try_fmt,
>>> + .vidioc_try_fmt_vid_out = bcm2835_isp_node_try_fmt,
>>> + .vidioc_try_fmt_meta_cap = bcm2835_isp_node_try_fmt,
>>> + .vidioc_s_selection = bcm2835_isp_node_s_selection,
>>> + .vidioc_g_selection = bcm2835_isp_node_g_selection,
>>> +
>>> + .vidioc_enum_fmt_vid_cap = bcm2835_isp_node_enum_fmt,
>>> + .vidioc_enum_fmt_vid_out = bcm2835_isp_node_enum_fmt,
>>> + .vidioc_enum_fmt_meta_cap = bcm2835_isp_node_enum_fmt,
>>> + .vidioc_enum_framesizes = bcm2835_isp_enum_framesizes,
>>> +
>>> + .vidioc_reqbufs = vb2_ioctl_reqbufs,
>>> + .vidioc_querybuf = vb2_ioctl_querybuf,
>>> + .vidioc_qbuf = vb2_ioctl_qbuf,
>>> + .vidioc_dqbuf = vb2_ioctl_dqbuf,
>>> + .vidioc_expbuf = vb2_ioctl_expbuf,
>>> + .vidioc_create_bufs = vb2_ioctl_create_bufs,
>>> + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
>>> +
>>> + .vidioc_streamon = vb2_ioctl_streamon,
>>> + .vidioc_streamoff = vb2_ioctl_streamoff,
>>> +
>>> + .vidioc_subscribe_event = bcm3285_isp_subscribe_event,
>>> + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
>>> +};
>>> +
>>> +/*
>>> + * Size of the array to provide to the VPU when asking for the list of supported
>>> + * formats.
>>> + *
>>> + * The ISP component currently advertises 33 input formats, so add a small
>>> + * overhead on that.
>>> + */
>>> +#define MAX_SUPPORTED_ENCODINGS 40
>>> +
>>> +/* Populate node->supported_fmts with the formats supported by those ports. */
>>> +static int bcm2835_isp_get_supported_fmts(struct bcm2835_isp_node *node)
>>> +{
>>> + struct bcm2835_isp_dev *dev = node_get_dev(node);
>>> + struct bcm2835_isp_fmt const **list;
>>> + unsigned int i, j, num_encodings;
>>> + u32 fourccs[MAX_SUPPORTED_ENCODINGS];
>>> + u32 param_size = sizeof(fourccs);
>>> + int ret;
>>> +
>>> + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance,
>>> + get_port_data(node),
>>> + MMAL_PARAMETER_SUPPORTED_ENCODINGS,
>>> + &fourccs, ¶m_size);
>>> +
>>> + if (ret) {
>>> + if (ret == MMAL_MSG_STATUS_ENOSPC) {
>>> + v4l2_err(&dev->v4l2_dev,
>>> + "%s: port has more encoding than we provided space for. Some are dropped.\n",
>>> + __func__);
>>> + num_encodings = MAX_SUPPORTED_ENCODINGS;
>>> + } else {
>>> + v4l2_err(&dev->v4l2_dev, "%s: get_param ret %u.\n",
>>> + __func__, ret);
>>> + return -EINVAL;
>>> + }
>>> + } else {
>>> + num_encodings = param_size / sizeof(u32);
>>> + }
>>> +
>>> + /*
>>> + * Assume at this stage that all encodings will be supported in V4L2.
>>> + * Any that aren't supported will waste a very small amount of memory.
>>> + */
>>> + list = devm_kzalloc(dev->dev,
>>> + sizeof(struct bcm2835_isp_fmt *) * num_encodings,
>>> + GFP_KERNEL);
>>> + if (!list)
>>> + return -ENOMEM;
>>> + node->supported_fmts.list = list;
>>> +
>>> + for (i = 0, j = 0; i < num_encodings; i++) {
>>> + const struct bcm2835_isp_fmt *fmt = get_fmt(fourccs[i]);
>>> +
>>> + if (fmt) {
>>> + list[j] = fmt;
>>> + j++;
>>> + }
>>> + }
>>> + node->supported_fmts.num_entries = j;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +/*
>>> + * Register a device node /dev/video<N> to go along with one of the ISP's input
>>> + * or output nodes.
>>> + */
>>> +static int register_node(struct bcm2835_isp_dev *dev,
>>> + struct bcm2835_isp_node *node,
>>> + int index)
>>> +{
>>> + struct video_device *vfd;
>>> + struct vb2_queue *queue;
>>> + int ret;
>>> +
>>> + mutex_init(&node->lock);
>>> +
>>> + node->dev = dev;
>>> + vfd = &node->vfd;
>>> + queue = &node->queue;
>>> + queue->type = index_to_queue_type(index);
>>> + /*
>>> + * Setup the node type-specific params.
>>> + *
>>> + * Only the OUTPUT node can set controls and crop windows. However,
>>> + * we must allow the s/g_selection ioctl on the stats node as v4l2
>>> + * compliance expects it to return a -ENOTTY, and the framework
>>> + * does not handle it if the ioctl is disabled.
>>> + */
>>> + switch (queue->type) {
>>> + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
>>> + vfd->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
>>> + node->id = index;
>>> + node->vfl_dir = VFL_DIR_TX;
>>> + node->name = "output";
>>> + break;
>>> + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
>>> + vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
>>> + /* First Capture node starts at id 0, etc. */
>>> + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
>>> + node->vfl_dir = VFL_DIR_RX;
>>> + node->name = "capture";
>>> + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
>>> + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_SELECTION);
>>> + v4l2_disable_ioctl(&node->vfd, VIDIOC_G_SELECTION);
>>> + break;
>>> + case V4L2_BUF_TYPE_META_CAPTURE:
>>> + vfd->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
>>> + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
>>> + node->vfl_dir = VFL_DIR_RX;
>>> + node->name = "stats";
>>> + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
>>
>> Why not disable S/G_SELECTION for meta capture here rather than test for it
>> in the op functions?
>
> I'd have to ask Naush. There were some very odd interactions with
> v4l2-compliance that he was struggling with.
>
>>> + break;
>>> + }
>>> +
>>> + /* We use the selection API instead of the old crop API. */
>>> + v4l2_disable_ioctl(vfd, VIDIOC_CROPCAP);
>>> + v4l2_disable_ioctl(vfd, VIDIOC_G_CROP);
>>> + v4l2_disable_ioctl(vfd, VIDIOC_S_CROP);
>>
>> No need for this: the core handles this and will disable these ioctls
>> automatically.
>
> The core will ENABLE these if g_selection and s_selection are defined,
> and uses mapping functions to try and convert between the two APIs.
> This may be down to missing the target check in s_selection, but again
> I seem to recall v4l2-compliance was reporting failures on these
> ioctls.
I suspect that the missing target check in g/s_selection is the cause of
the v4l2-compliance failures.
If there are still compliance issues after adding that check, then just
reach out to me for help.
Regards,
Hans
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-18 12:02 ` Hans Verkuil
2020-05-18 14:36 ` Dave Stevenson
@ 2020-05-19 12:47 ` Naushir Patuck
2020-05-19 13:03 ` Jacopo Mondi
2020-06-24 11:28 ` Hans Verkuil
1 sibling, 2 replies; 111+ messages in thread
From: Naushir Patuck @ 2020-05-19 12:47 UTC (permalink / raw)
To: Hans Verkuil
Cc: Laurent Pinchart, linux-media, Kieran Bingham, Jacopo Mondi,
Niklas Söderlund, Dave Stevenson
Hi Hans,
Thank you for the review and comments. Just a follow up and
overlapping Dave's comments:
On Mon, 18 May 2020 at 13:02, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 04/05/2020 11:26, Laurent Pinchart wrote:
> > From: Naushir Patuck <naush@raspberrypi.com>
> >
> > Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
> > chips.
> >
> > The driver interfaces though the VideoCore unit using the VCHIQ MMAL
> > interface.
> >
> > ISP driver upported from from RaspberryPi BSP at revision:
> > 6c3505be6c3e ("staging: vc04_services: isp: Make all references to bcm2835_isp_fmt const")
> >
> > Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
> > Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> > [Adapt to staging by moving all modifications that in the BSP are scattered
> > in core components inside this directory]
> > Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> > ---
> > .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 41 +
> > drivers/staging/vc04_services/Kconfig | 2 +
> > drivers/staging/vc04_services/Makefile | 1 +
> > .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> > .../vc04_services/bcm2835-isp/Makefile | 10 +
> > .../bcm2835-isp/bcm2835-v4l2-isp.c | 1632 +++++++++++++++++
> > .../bcm2835-isp/bcm2835_isp_ctrls.h | 67 +
> > .../bcm2835-isp/bcm2835_isp_fmts.h | 301 +++
> > .../include/uapi/linux/bcm2835-isp.h | 333 ++++
> > .../staging/vc04_services/vchiq-mmal/Kconfig | 3 +-
> > .../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
> > .../vchiq-mmal/mmal-parameters.h | 153 +-
> > 12 files changed, 2559 insertions(+), 2 deletions(-)
> > create mode 100644 drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> > create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> > create mode 100644 drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> >
>
> <snip>
>
> > diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> > new file mode 100644
> > index 000000000000..a32faab4b8dc
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
> > @@ -0,0 +1,1632 @@
>
> <snip>
>
> > +// SPDX-License-Identifier: GPL-2.0
> > +/*
> > + * Broadcom BCM2835 ISP driver
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/platform_device.h>
> > +
> > +#include <media/v4l2-ctrls.h>
> > +#include <media/v4l2-device.h>
> > +#include <media/v4l2-event.h>
> > +#include <media/v4l2-ioctl.h>
> > +#include <media/videobuf2-dma-contig.h>
> > +
> > +#include "vchiq-mmal/mmal-msg.h"
> > +#include "vchiq-mmal/mmal-parameters.h"
> > +#include "vchiq-mmal/mmal-vchiq.h"
> > +
> > +#include "bcm2835_isp_ctrls.h"
> > +#include "bcm2835_isp_fmts.h"
> > +
> > +static unsigned int debug;
> > +module_param(debug, uint, 0644);
> > +MODULE_PARM_DESC(debug, "activates debug info");
> > +
> > +static unsigned int video_nr = 13;
> > +module_param(video_nr, uint, 0644);
> > +MODULE_PARM_DESC(video_nr, "base video device number");
> > +
> > +#define BCM2835_ISP_NAME "bcm2835-isp"
> > +#define BCM2835_ISP_ENTITY_NAME_LEN 32
> > +
> > +#define BCM2835_ISP_NUM_OUTPUTS 1
> > +#define BCM2835_ISP_NUM_CAPTURES 2
> > +#define BCM2835_ISP_NUM_METADATA 1
> > +
> > +#define BCM2835_ISP_NUM_NODES \
> > + (BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES + \
> > + BCM2835_ISP_NUM_METADATA)
> > +
> > +/* Default frame dimension of 1280 pixels. */
> > +#define DEFAULT_DIM 1280U
> > +/*
> > + * Maximum frame dimension of 16384 pixels. Even though the ISP runs in tiles,
> > + * have a sensible limit so that we do not create an excessive number of tiles
> > + * to process.
> > + */
> > +#define MAX_DIM 16384U
> > +/*
> > + * Minimum frame dimension of 64 pixels. Anything lower, and the tiling
> > + * algorihtm may not be able to cope when applying filter context.
> > + */
> > +#define MIN_DIM 64U
> > +
> > +/* Per-queue, driver-specific private data */
> > +struct bcm2835_isp_q_data {
> > + /*
> > + * These parameters should be treated as gospel, with everything else
> > + * being determined from them.
> > + */
> > + unsigned int bytesperline;
> > + unsigned int width;
> > + unsigned int height;
> > + unsigned int sizeimage;
> > + const struct bcm2835_isp_fmt *fmt;
> > +};
> > +
> > +/*
> > + * Structure to describe a single node /dev/video<N> which represents a single
> > + * input or output queue to the ISP device.
> > + */
> > +struct bcm2835_isp_node {
> > + int vfl_dir;
> > + unsigned int id;
> > + const char *name;
> > + struct video_device vfd;
> > + struct media_pad pad;
> > + struct media_intf_devnode *intf_devnode;
> > + struct media_link *intf_link;
> > + struct mutex lock; /* top level device node lock */
> > + struct mutex queue_lock;
> > +
> > + struct vb2_queue queue;
> > + unsigned int sequence;
> > +
> > + /* The list of formats supported on the node. */
> > + struct bcm2835_isp_fmt_list supported_fmts;
> > +
> > + struct bcm2835_isp_q_data q_data;
> > +
> > + /* Parent device structure */
> > + struct bcm2835_isp_dev *dev;
> > +
> > + bool registered;
> > + bool media_node_registered;
> > + bool queue_init;
> > +};
> > +
> > +/*
> > + * Structure representing the entire ISP device, comprising several input and
> > + * output nodes /dev/video<N>.
> > + */
> > +struct bcm2835_isp_dev {
> > + struct v4l2_device v4l2_dev;
> > + struct device *dev;
> > + struct v4l2_ctrl_handler ctrl_handler;
> > + struct media_device mdev;
> > + struct media_entity entity;
> > + bool media_device_registered;
> > + bool media_entity_registered;
> > + struct vchiq_mmal_instance *mmal_instance;
> > + struct vchiq_mmal_component *component;
> > + struct completion frame_cmplt;
> > +
> > + struct bcm2835_isp_node node[BCM2835_ISP_NUM_NODES];
> > + struct media_pad pad[BCM2835_ISP_NUM_NODES];
> > + atomic_t num_streaming;
> > +
> > + /* Image pipeline controls. */
> > + int r_gain;
> > + int b_gain;
> > +};
> > +
> > +struct bcm2835_isp_buffer {
> > + struct vb2_v4l2_buffer vb;
> > + struct mmal_buffer mmal;
> > +};
> > +
> > +static
> > +inline struct bcm2835_isp_dev *node_get_dev(struct bcm2835_isp_node *node)
> > +{
> > + return node->dev;
> > +}
> > +
> > +static inline bool node_is_output(struct bcm2835_isp_node *node)
> > +{
> > + return node->queue.type == V4L2_BUF_TYPE_VIDEO_OUTPUT;
> > +}
> > +
> > +static inline bool node_is_capture(struct bcm2835_isp_node *node)
> > +{
> > + return node->queue.type == V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > +}
> > +
> > +static inline bool node_is_stats(struct bcm2835_isp_node *node)
> > +{
> > + return node->queue.type == V4L2_BUF_TYPE_META_CAPTURE;
> > +}
> > +
> > +static inline enum v4l2_buf_type index_to_queue_type(int index)
> > +{
> > + if (index < BCM2835_ISP_NUM_OUTPUTS)
> > + return V4L2_BUF_TYPE_VIDEO_OUTPUT;
> > + else if (index < BCM2835_ISP_NUM_OUTPUTS + BCM2835_ISP_NUM_CAPTURES)
> > + return V4L2_BUF_TYPE_VIDEO_CAPTURE;
> > + else
> > + return V4L2_BUF_TYPE_META_CAPTURE;
> > +}
> > +
> > +static struct vchiq_mmal_port *get_port_data(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + if (!dev->component)
> > + return NULL;
> > +
> > + switch (node->queue.type) {
> > + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> > + return &dev->component->input[node->id];
> > + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> > + case V4L2_BUF_TYPE_META_CAPTURE:
> > + return &dev->component->output[node->id];
> > + default:
> > + v4l2_err(&dev->v4l2_dev, "%s: Invalid queue type %u\n",
> > + __func__, node->queue.type);
> > + break;
> > + }
> > + return NULL;
> > +}
> > +
> > +static int set_isp_param(struct bcm2835_isp_node *node, u32 parameter,
> > + void *value, u32 value_size)
> > +{
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> > + parameter, value, value_size);
> > +}
> > +
> > +static int set_wb_gains(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct mmal_parameter_awbgains gains = {
> > + .r_gain = { dev->r_gain, 1000 },
> > + .b_gain = { dev->b_gain, 1000 }
> > + };
> > +
> > + return set_isp_param(node, MMAL_PARAMETER_CUSTOM_AWB_GAINS,
> > + &gains, sizeof(gains));
> > +}
> > +
> > +static int set_digital_gain(struct bcm2835_isp_node *node, uint32_t gain)
> > +{
> > + struct mmal_parameter_rational digital_gain = {
> > + .num = gain,
> > + .den = 1000
> > + };
> > +
> > + return set_isp_param(node, MMAL_PARAMETER_DIGITAL_GAIN,
> > + &digital_gain, sizeof(digital_gain));
> > +}
> > +
> > +static const struct bcm2835_isp_fmt *get_fmt(u32 mmal_fmt)
> > +{
> > + unsigned int i;
> > +
> > + for (i = 0; i < ARRAY_SIZE(supported_formats); i++) {
> > + if (supported_formats[i].mmal_fmt == mmal_fmt)
> > + return &supported_formats[i];
> > + }
> > + return NULL;
> > +}
> > +
> > +static const
> > +struct bcm2835_isp_fmt *find_format_by_fourcc(unsigned int fourcc,
> > + struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
> > + const struct bcm2835_isp_fmt *fmt;
> > + unsigned int i;
> > +
> > + for (i = 0; i < fmts->num_entries; i++) {
> > + fmt = fmts->list[i];
> > + if (fmt->fourcc == fourcc)
> > + return fmt;
> > + }
> > +
> > + return NULL;
> > +}
> > +
> > +static const
> > +struct bcm2835_isp_fmt *find_format(struct v4l2_format *f,
> > + struct bcm2835_isp_node *node)
> > +{
> > + return find_format_by_fourcc(node_is_stats(node) ?
> > + f->fmt.meta.dataformat :
> > + f->fmt.pix.pixelformat,
> > + node);
> > +}
> > +
> > +/* vb2_to_mmal_buffer() - converts vb2 buffer header to MMAL
> > + *
> > + * Copies all the required fields from a VB2 buffer to the MMAL buffer header,
> > + * ready for sending to the VPU.
> > + */
> > +static void vb2_to_mmal_buffer(struct mmal_buffer *buf,
> > + struct vb2_v4l2_buffer *vb2)
> > +{
> > + u64 pts;
> > +
> > + buf->mmal_flags = 0;
> > + if (vb2->flags & V4L2_BUF_FLAG_KEYFRAME)
> > + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_KEYFRAME;
> > +
> > + /* Data must be framed correctly as one frame per buffer. */
> > + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_FRAME_END;
> > +
> > + buf->length = vb2->vb2_buf.planes[0].bytesused;
> > + /*
> > + * Minor ambiguity in the V4L2 spec as to whether passing in a 0 length
> > + * buffer, or one with V4L2_BUF_FLAG_LAST set denotes end of stream.
> > + * Handle either.
> > + */
> > + if (!buf->length || vb2->flags & V4L2_BUF_FLAG_LAST)
> > + buf->mmal_flags |= MMAL_BUFFER_HEADER_FLAG_EOS;
> > +
> > + /* vb2 timestamps in nsecs, mmal in usecs */
> > + pts = vb2->vb2_buf.timestamp;
> > + do_div(pts, 1000);
> > + buf->pts = pts;
> > + buf->dts = MMAL_TIME_UNKNOWN;
> > +}
> > +
> > +static void mmal_buffer_cb(struct vchiq_mmal_instance *instance,
> > + struct vchiq_mmal_port *port, int status,
> > + struct mmal_buffer *mmal_buf)
> > +{
> > + struct bcm2835_isp_buffer *q_buf;
> > + struct bcm2835_isp_node *node = port->cb_ctx;
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vb2_v4l2_buffer *vb2;
> > +
> > + q_buf = container_of(mmal_buf, struct bcm2835_isp_buffer, mmal);
> > + vb2 = &q_buf->vb;
> > + v4l2_dbg(2, debug, &dev->v4l2_dev,
> > + "%s: port:%s[%d], status:%d, buf:%p, dmabuf:%p, length:%lu, flags %u, pts %lld\n",
> > + __func__, node_is_output(node) ? "input" : "output", node->id,
> > + status, mmal_buf, mmal_buf->dma_buf, mmal_buf->length,
> > + mmal_buf->mmal_flags, mmal_buf->pts);
> > +
> > + if (mmal_buf->cmd)
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Unexpected event on output callback - %08x\n",
> > + __func__, mmal_buf->cmd);
> > +
> > + if (status) {
> > + /* error in transfer */
> > + if (vb2) {
> > + /* there was a buffer with the error so return it */
> > + vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_ERROR);
> > + }
> > + return;
> > + }
> > +
> > + /* vb2 timestamps in nsecs, mmal in usecs */
> > + vb2->vb2_buf.timestamp = mmal_buf->pts * 1000;
> > + vb2->sequence = node->sequence++;
> > + vb2_set_plane_payload(&vb2->vb2_buf, 0, mmal_buf->length);
> > + vb2_buffer_done(&vb2->vb2_buf, VB2_BUF_STATE_DONE);
> > +
> > + if (!port->enabled)
> > + complete(&dev->frame_cmplt);
> > +}
> > +
> > +static void setup_mmal_port_format(struct bcm2835_isp_node *node,
> > + struct vchiq_mmal_port *port)
> > +{
> > + struct bcm2835_isp_q_data *q_data = &node->q_data;
> > +
> > + port->format.encoding = q_data->fmt->mmal_fmt;
> > + /* Raw image format - set width/height */
> > + port->es.video.width = (q_data->bytesperline << 3) / q_data->fmt->depth;
> > + port->es.video.height = q_data->height;
> > + port->es.video.crop.width = q_data->width;
> > + port->es.video.crop.height = q_data->height;
> > + port->es.video.crop.x = 0;
> > + port->es.video.crop.y = 0;
> > +};
> > +
> > +static int setup_mmal_port(struct bcm2835_isp_node *node)
> > +{
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + unsigned int enable = 1;
> > + int ret;
> > +
> > + v4l2_dbg(2, debug, &dev->v4l2_dev, "%s: setup %s[%d]\n", __func__,
> > + node->name, node->id);
> > +
> > + vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> > + MMAL_PARAMETER_ZERO_COPY, &enable,
> > + sizeof(enable));
> > + setup_mmal_port_format(node, port);
> > + ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
> > + if (ret < 0) {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: vchiq_mmal_port_set_format failed\n",
> > + __func__);
> > + return ret;
> > + }
> > +
> > + if (node->q_data.sizeimage < port->minimum_buffer.size) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "buffer size mismatch sizeimage %u < min size %u\n",
> > + node->q_data.sizeimage, port->minimum_buffer.size);
> > + return -EINVAL;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_mmal_buf_cleanup(struct mmal_buffer *mmal_buf)
> > +{
> > + mmal_vchi_buffer_cleanup(mmal_buf);
> > +
> > + if (mmal_buf->dma_buf) {
> > + dma_buf_put(mmal_buf->dma_buf);
> > + mmal_buf->dma_buf = NULL;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_queue_setup(struct vb2_queue *q,
> > + unsigned int *nbuffers,
> > + unsigned int *nplanes,
> > + unsigned int sizes[],
> > + struct device *alloc_devs[])
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> > + struct vchiq_mmal_port *port;
> > + unsigned int size;
> > +
> > + if (setup_mmal_port(node))
> > + return -EINVAL;
> > +
> > + size = node->q_data.sizeimage;
> > + if (size == 0) {
> > + v4l2_info(&node_get_dev(node)->v4l2_dev,
> > + "%s: Image size unset in queue_setup for node %s[%d]\n",
> > + __func__, node->name, node->id);
> > + return -EINVAL;
> > + }
> > +
> > + if (*nplanes)
> > + return sizes[0] < size ? -EINVAL : 0;
> > +
> > + *nplanes = 1;
> > + sizes[0] = size;
> > +
> > + port = get_port_data(node);
> > + port->current_buffer.size = size;
> > +
> > + if (*nbuffers < port->minimum_buffer.num)
> > + *nbuffers = port->minimum_buffer.num;
> > +
> > + port->current_buffer.num = *nbuffers;
> > +
> > + v4l2_dbg(2, debug, &node_get_dev(node)->v4l2_dev,
> > + "%s: Image size %u, nbuffers %u for node %s[%d]\n",
> > + __func__, sizes[0], *nbuffers, node->name, node->id);
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_buf_init(struct vb2_buffer *vb)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> > + struct bcm2835_isp_buffer *buf =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > +
> > + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: vb %p\n", __func__, vb);
> > +
> > + buf->mmal.buffer = vb2_plane_vaddr(&buf->vb.vb2_buf, 0);
> > + buf->mmal.buffer_size = vb2_plane_size(&buf->vb.vb2_buf, 0);
> > + mmal_vchi_buffer_init(dev->mmal_instance, &buf->mmal);
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_buf_prepare(struct vb2_buffer *vb)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(vb->vb2_queue);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> > + struct bcm2835_isp_buffer *buf =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > + struct dma_buf *dma_buf;
> > + int ret;
> > +
> > + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: type: %d ptr %p\n",
> > + __func__, vb->vb2_queue->type, vb);
> > +
> > + if (V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type)) {
> > + if (vb2->field == V4L2_FIELD_ANY)
> > + vb2->field = V4L2_FIELD_NONE;
> > + if (vb2->field != V4L2_FIELD_NONE) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s field isn't supported\n", __func__);
> > + return -EINVAL;
> > + }
> > + }
> > +
> > + if (vb2_plane_size(vb, 0) < node->q_data.sizeimage) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s data will not fit into plane (%lu < %lu)\n",
> > + __func__, vb2_plane_size(vb, 0),
> > + (long)node->q_data.sizeimage);
> > + return -EINVAL;
> > + }
> > +
> > + if (!V4L2_TYPE_IS_OUTPUT(vb->vb2_queue->type))
> > + vb2_set_plane_payload(vb, 0, node->q_data.sizeimage);
> > +
> > + switch (vb->memory) {
> > + case VB2_MEMORY_DMABUF:
> > + dma_buf = dma_buf_get(vb->planes[0].m.fd);
> > +
> > + if (dma_buf != buf->mmal.dma_buf) {
> > + /*
> > + * dmabuf either hasn't already been mapped, or it has
> > + * changed.
> > + */
> > + if (buf->mmal.dma_buf) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s Buffer changed - why did the core not call cleanup?\n",
> > + __func__);
> > + bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
> > + }
> > +
> > + buf->mmal.dma_buf = dma_buf;
> > + } else {
> > + /*
> > + * Already have a reference to the buffer, so release it
> > + * here.
> > + */
> > + dma_buf_put(dma_buf);
> > + }
> > + ret = 0;
> > + break;
> > + case VB2_MEMORY_MMAP:
> > + /*
> > + * We want to do this at init, but vb2_core_expbuf checks that
> > + * the index < q->num_buffers, and q->num_buffers only gets
> > + * updated once all the buffers are allocated.
> > + */
> > + if (!buf->mmal.dma_buf) {
> > + ret = vb2_core_expbuf_dmabuf(vb->vb2_queue,
> > + vb->vb2_queue->type,
> > + vb->index, 0, O_CLOEXEC,
> > + &buf->mmal.dma_buf);
> > + v4l2_dbg(3, debug, &dev->v4l2_dev,
> > + "%s: exporting ptr %p to dmabuf %p\n",
> > + __func__, vb, buf->mmal.dma_buf);
> > + if (ret)
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed to expbuf idx %d, ret %d\n",
> > + __func__, vb->index, ret);
> > + } else {
> > + ret = 0;
> > + }
> > + break;
> > + default:
> > + ret = -EINVAL;
> > + break;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static void bcm2835_isp_node_buffer_queue(struct vb2_buffer *buf)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(buf->vb2_queue);
> > + struct vb2_v4l2_buffer *vbuf =
> > + container_of(buf, struct vb2_v4l2_buffer, vb2_buf);
> > + struct bcm2835_isp_buffer *buffer =
> > + container_of(vbuf, struct bcm2835_isp_buffer, vb);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + v4l2_dbg(3, debug, &dev->v4l2_dev, "%s: node %s[%d], buffer %p\n",
> > + __func__, node->name, node->id, buffer);
> > +
> > + vb2_to_mmal_buffer(&buffer->mmal, &buffer->vb);
> > + v4l2_dbg(3, debug, &dev->v4l2_dev,
> > + "%s: node %s[%d] - submitting mmal dmabuf %p\n", __func__,
> > + node->name, node->id, buffer->mmal.dma_buf);
> > + vchiq_mmal_submit_buffer(dev->mmal_instance, get_port_data(node),
> > + &buffer->mmal);
> > +}
> > +
> > +static void bcm2835_isp_buffer_cleanup(struct vb2_buffer *vb)
> > +{
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(vb);
> > + struct bcm2835_isp_buffer *buffer =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > +
> > + bcm2835_isp_mmal_buf_cleanup(&buffer->mmal);
> > +}
> > +
> > +static int bcm2835_isp_node_start_streaming(struct vb2_queue *q,
> > + unsigned int count)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + int ret;
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d] (count %u)\n",
> > + __func__, node->name, node->id, count);
> > +
> > + ret = vchiq_mmal_component_enable(dev->mmal_instance, dev->component);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev, "%s: Failed enabling component, ret %d\n",
> > + __func__, ret);
> > + return -EIO;
> > + }
> > +
> > + node->sequence = 0;
> > + port->cb_ctx = node;
> > + ret = vchiq_mmal_port_enable(dev->mmal_instance, port,
> > + mmal_buffer_cb);
> > + if (!ret)
> > + atomic_inc(&dev->num_streaming);
> > + else
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed enabling port, ret %d\n", __func__, ret);
> > +
>
> It's not obvious from this code what happens with outstanding buffers if there
> is an error. They should be returned to vb2 with state VB2_BUF_STATE_QUEUED.
> Does that happen?
>
> > + return ret;
> > +}
> > +
> > +static void bcm2835_isp_node_stop_streaming(struct vb2_queue *q)
> > +{
> > + struct bcm2835_isp_node *node = vb2_get_drv_priv(q);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + unsigned int i;
> > + int ret;
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev, "%s: node %s[%d], mmal port %p\n",
> > + __func__, node->name, node->id, port);
> > +
> > + init_completion(&dev->frame_cmplt);
> > +
> > + /* Disable MMAL port - this will flush buffers back */
> > + ret = vchiq_mmal_port_disable(dev->mmal_instance, port);
> > + if (ret)
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed disabling %s port, ret %d\n", __func__,
> > + node_is_output(node) ? "i/p" : "o/p",
> > + ret);
> > +
> > + while (atomic_read(&port->buffers_with_vpu)) {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Waiting for buffers to be returned - %d outstanding\n",
> > + __func__, atomic_read(&port->buffers_with_vpu));
> > + ret = wait_for_completion_timeout(&dev->frame_cmplt, HZ);
> > + if (ret <= 0) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Timeout waiting for buffers to be returned - %d outstanding\n",
> > + __func__,
> > + atomic_read(&port->buffers_with_vpu));
> > + break;
> > + }
> > + }
> > +
> > + /* Release the VCSM handle here to release the associated dmabuf */
> > + for (i = 0; i < q->num_buffers; i++) {
> > + struct vb2_v4l2_buffer *vb2 = to_vb2_v4l2_buffer(q->bufs[i]);
> > + struct bcm2835_isp_buffer *buf =
> > + container_of(vb2, struct bcm2835_isp_buffer, vb);
> > + bcm2835_isp_mmal_buf_cleanup(&buf->mmal);
> > + }
> > +
> > + atomic_dec(&dev->num_streaming);
> > + /* If all ports disabled, then disable the component */
> > + if (atomic_read(&dev->num_streaming) == 0) {
> > + ret = vchiq_mmal_component_disable(dev->mmal_instance,
> > + dev->component);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed disabling component, ret %d\n",
> > + __func__, ret);
> > + }
> > + }
> > +
> > + /*
> > + * Simply wait for any vb2 buffers to finish. We could take steps to
> > + * make them complete more quickly if we care, or even return them
> > + * ourselves.
> > + */
> > + vb2_wait_for_all_buffers(&node->queue);
> > +}
> > +
> > +static const struct vb2_ops bcm2835_isp_node_queue_ops = {
> > + .queue_setup = bcm2835_isp_node_queue_setup,
> > + .buf_init = bcm2835_isp_buf_init,
> > + .buf_prepare = bcm2835_isp_buf_prepare,
> > + .buf_queue = bcm2835_isp_node_buffer_queue,
> > + .buf_cleanup = bcm2835_isp_buffer_cleanup,
> > + .start_streaming = bcm2835_isp_node_start_streaming,
> > + .stop_streaming = bcm2835_isp_node_stop_streaming,
> > +};
> > +
> > +static const
> > +struct bcm2835_isp_fmt *get_default_format(struct bcm2835_isp_node *node)
> > +{
> > + return node->supported_fmts.list[0];
> > +}
> > +
> > +static inline unsigned int get_bytesperline(int width,
> > + const struct bcm2835_isp_fmt *fmt)
> > +{
> > + return ALIGN((width * fmt->depth) >> 3, fmt->bytesperline_align);
> > +}
> > +
> > +static inline unsigned int get_sizeimage(int bpl, int width, int height,
> > + const struct bcm2835_isp_fmt *fmt)
> > +{
> > + return (bpl * height * fmt->size_multiplier_x2) >> 1;
> > +}
> > +
> > +static int bcm2835_isp_s_ctrl(struct v4l2_ctrl *ctrl)
> > +{
> > + struct bcm2835_isp_dev *dev =
> > + container_of(ctrl->handler, struct bcm2835_isp_dev, ctrl_handler);
> > + struct bcm2835_isp_node *node = &dev->node[0];
> > + int ret = 0;
> > +
> > + /*
> > + * The ISP firmware driver will ensure these settings are applied on
> > + * a frame boundary, so we are safe to write them as they come in.
> > + *
> > + * Note that the bcm2835_isp_* param structures are identical to the
> > + * mmal-parameters.h definitions. This avoids the need for unnecessary
> > + * field-by-field copying between structures.
> > + */
> > + switch (ctrl->id) {
> > + case V4L2_CID_RED_BALANCE:
> > + dev->r_gain = ctrl->val;
> > + ret = set_wb_gains(node);
> > + break;
> > + case V4L2_CID_BLUE_BALANCE:
> > + dev->b_gain = ctrl->val;
> > + ret = set_wb_gains(node);
> > + break;
> > + case V4L2_CID_DIGITAL_GAIN:
> > + ret = set_digital_gain(node, ctrl->val);
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_CC_MATRIX:
> > + ret = set_isp_param(node, MMAL_PARAMETER_CUSTOM_CCM,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_custom_ccm));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_LENS_SHADING:
> > + ret = set_isp_param(node, MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_lens_shading));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL:
> > + ret = set_isp_param(node, MMAL_PARAMETER_BLACK_LEVEL,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_black_level));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_GEQ:
> > + ret = set_isp_param(node, MMAL_PARAMETER_GEQ,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_geq));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_GAMMA:
> > + ret = set_isp_param(node, MMAL_PARAMETER_GAMMA,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_gamma));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_DENOISE:
> > + ret = set_isp_param(node, MMAL_PARAMETER_DENOISE,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_denoise));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_SHARPEN:
> > + ret = set_isp_param(node, MMAL_PARAMETER_SHARPEN,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_sharpen));
> > + break;
> > + case V4L2_CID_USER_BCM2835_ISP_DPC:
> > + ret = set_isp_param(node, MMAL_PARAMETER_DPC,
> > + ctrl->p_new.p_u8,
> > + sizeof(struct bcm2835_isp_dpc));
> > + break;
> > + default:
> > + v4l2_info(&dev->v4l2_dev, "Unrecognised control\n");
> > + ret = -EINVAL;
> > + }
> > +
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev, "%s: Failed setting ctrl \"%s\" (%08x), err %d\n",
> > + __func__, ctrl->name, ctrl->id, ret);
> > + ret = -EIO;
> > + }
> > +
> > + return ret;
> > +}
> > +
> > +static const struct v4l2_ctrl_ops bcm2835_isp_ctrl_ops = {
> > + .s_ctrl = bcm2835_isp_s_ctrl,
> > +};
> > +
> > +static const struct v4l2_file_operations bcm2835_isp_fops = {
> > + .owner = THIS_MODULE,
> > + .open = v4l2_fh_open,
> > + .release = vb2_fop_release,
> > + .poll = vb2_fop_poll,
> > + .unlocked_ioctl = video_ioctl2,
> > + .mmap = vb2_fop_mmap
> > +};
> > +
> > +static int populate_qdata_fmt(struct v4l2_format *f,
> > + struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct bcm2835_isp_q_data *q_data = &node->q_data;
> > + struct vchiq_mmal_port *port;
> > + int ret;
> > +
> > + if (!node_is_stats(node)) {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Setting pix format for type %d, wxh: %ux%u, fmt: %08x, size %u\n",
> > + __func__, f->type, f->fmt.pix.width, f->fmt.pix.height,
> > + f->fmt.pix.pixelformat, f->fmt.pix.sizeimage);
> > +
> > + q_data->fmt = find_format(f, node);
> > + q_data->width = f->fmt.pix.width;
> > + q_data->height = f->fmt.pix.height;
> > + q_data->height = f->fmt.pix.height;
> > +
> > + /* All parameters should have been set correctly by try_fmt */
> > + q_data->bytesperline = f->fmt.pix.bytesperline;
> > + q_data->sizeimage = f->fmt.pix.sizeimage;
> > + } else {
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Setting meta format for fmt: %08x, size %u\n",
> > + __func__, f->fmt.meta.dataformat,
> > + f->fmt.meta.buffersize);
> > +
> > + q_data->fmt = find_format(f, node);
> > + q_data->width = 0;
> > + q_data->height = 0;
> > + q_data->bytesperline = 0;
> > + q_data->sizeimage = f->fmt.meta.buffersize;
> > + }
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Calculated bpl as %u, size %u\n", __func__,
> > + q_data->bytesperline, q_data->sizeimage);
> > +
> > + /* If we have a component then setup the port as well */
> > + port = get_port_data(node);
> > + setup_mmal_port_format(node, port);
> > + ret = vchiq_mmal_port_set_format(dev->mmal_instance, port);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Failed vchiq_mmal_port_set_format on port, ret %d\n",
> > + __func__, ret);
> > + ret = -EINVAL;
> > + }
> > +
> > + if (q_data->sizeimage < port->minimum_buffer.size) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: Current buffer size of %u < min buf size %u - driver mismatch to MMAL\n",
> > + __func__,
> > + q_data->sizeimage,
> > + port->minimum_buffer.size);
> > + }
> > +
> > + v4l2_dbg(1, debug, &dev->v4l2_dev,
> > + "%s: Set format for type %d, wxh: %dx%d, fmt: %08x, size %u\n",
> > + __func__, f->type, q_data->width, q_data->height,
> > + q_data->fmt->fourcc, q_data->sizeimage);
> > +
> > + return ret;
> > +}
> > +
> > +static int bcm2835_isp_node_querycap(struct file *file, void *priv,
> > + struct v4l2_capability *cap)
> > +{
> > + strscpy(cap->driver, BCM2835_ISP_NAME, sizeof(cap->driver));
> > + strscpy(cap->card, BCM2835_ISP_NAME, sizeof(cap->card));
> > + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s",
> > + BCM2835_ISP_NAME);
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_g_fmt(struct file *file, void *priv,
> > + struct v4l2_format *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + if (node_is_stats(node)) {
> > + f->fmt.meta.dataformat = V4L2_META_FMT_BCM2835_ISP_STATS;
> > + f->fmt.meta.buffersize =
> > + get_port_data(node)->minimum_buffer.size;
> > + } else {
> > + struct bcm2835_isp_q_data *q_data = &node->q_data;
> > +
> > + f->fmt.pix.width = q_data->width;
> > + f->fmt.pix.height = q_data->height;
> > + f->fmt.pix.field = V4L2_FIELD_NONE;
> > + f->fmt.pix.pixelformat = q_data->fmt->fourcc;
> > + f->fmt.pix.bytesperline = q_data->bytesperline;
> > + f->fmt.pix.sizeimage = q_data->sizeimage;
> > + f->fmt.pix.colorspace = q_data->fmt->colorspace;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_enum_fmt(struct file *file, void *priv,
> > + struct v4l2_fmtdesc *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_fmt_list *fmts = &node->supported_fmts;
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + if (f->index < fmts->num_entries) {
> > + /* Format found */
> > + f->pixelformat = fmts->list[f->index]->fourcc;
> > + f->flags = fmts->list[f->index]->flags;
> > + return 0;
> > + }
> > +
> > + return -EINVAL;
> > +}
> > +
> > +static int bcm2835_isp_enum_framesizes(struct file *file, void *priv,
> > + struct v4l2_frmsizeenum *fsize)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + const struct bcm2835_isp_fmt *fmt;
> > +
> > + if (node_is_stats(node) || fsize->index)
> > + return -EINVAL;
> > +
> > + fmt = find_format_by_fourcc(fsize->pixel_format, node);
> > + if (!fmt) {
> > + v4l2_err(&dev->v4l2_dev, "Invalid pixel code: %x\n",
> > + fsize->pixel_format);
> > + return -EINVAL;
> > + }
> > +
> > + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE;
> > + fsize->stepwise.min_width = MIN_DIM;
> > + fsize->stepwise.max_width = MAX_DIM;
> > + fsize->stepwise.step_width = fmt->step_size;
> > +
> > + fsize->stepwise.min_height = MIN_DIM;
> > + fsize->stepwise.max_height = MAX_DIM;
> > + fsize->stepwise.step_height = fmt->step_size;
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_try_fmt(struct file *file, void *priv,
> > + struct v4l2_format *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + const struct bcm2835_isp_fmt *fmt;
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + fmt = find_format(f, node);
> > + if (!fmt)
> > + fmt = get_default_format(node);
> > +
> > + if (!node_is_stats(node)) {
> > + f->fmt.pix.width = max(min(f->fmt.pix.width, MAX_DIM),
> > + MIN_DIM);
> > + f->fmt.pix.height = max(min(f->fmt.pix.height, MAX_DIM),
> > + MIN_DIM);
> > +
> > + f->fmt.pix.pixelformat = fmt->fourcc;
> > + f->fmt.pix.colorspace = fmt->colorspace;
> > + f->fmt.pix.bytesperline = get_bytesperline(f->fmt.pix.width,
> > + fmt);
> > + f->fmt.pix.field = V4L2_FIELD_NONE;
> > + f->fmt.pix.sizeimage =
> > + get_sizeimage(f->fmt.pix.bytesperline, f->fmt.pix.width,
> > + f->fmt.pix.height, fmt);
> > + } else {
> > + f->fmt.meta.dataformat = fmt->fourcc;
> > + f->fmt.meta.buffersize =
> > + get_port_data(node)->minimum_buffer.size;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_node_s_fmt(struct file *file, void *priv,
> > + struct v4l2_format *f)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + int ret;
> > +
> > + if (f->type != node->queue.type)
> > + return -EINVAL;
> > +
> > + ret = bcm2835_isp_node_try_fmt(file, priv, f);
> > + if (ret)
> > + return ret;
> > +
> > + v4l2_dbg(1, debug, &node_get_dev(node)->v4l2_dev,
> > + "%s: Set format for node %s[%d]\n",
> > + __func__, node->name, node->id);
> > +
> > + return populate_qdata_fmt(f, node);
> > +}
> > +
> > +static int bcm2835_isp_node_s_selection(struct file *file, void *fh,
> > + struct v4l2_selection *s)
> > +{
> > + struct mmal_parameter_crop crop;
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > +
> > + /* This return value is required fro V4L2 compliance. */
> > + if (node_is_stats(node))
> > + return -ENOTTY;
> > +
> > + if (!s->r.width || !s->r.height)
> > + return -EINVAL;
>
> I'm missing a check for s->target.
Ack
>
> > +
> > + /* Adjust the crop window if goes outside the frame dimensions. */
> > + s->r.left = min((unsigned int)max(s->r.left, 0),
> > + node->q_data.width - MIN_DIM);
> > + s->r.top = min((unsigned int)max(s->r.top, 0),
> > + node->q_data.height - MIN_DIM);
> > + s->r.width = max(min(s->r.width, node->q_data.width - s->r.left),
> > + MIN_DIM);
> > + s->r.height = max(min(s->r.height, node->q_data.height - s->r.top),
> > + MIN_DIM);
> > +
> > + crop.rect.x = s->r.left;
> > + crop.rect.y = s->r.top;
> > + crop.rect.width = s->r.width;
> > + crop.rect.height = s->r.height;
> > +
> > + return vchiq_mmal_port_parameter_set(dev->mmal_instance, port,
> > + MMAL_PARAMETER_CROP,
> > + &crop, sizeof(crop));
> > +}
> > +
> > +static int bcm2835_isp_node_g_selection(struct file *file, void *fh,
> > + struct v4l2_selection *s)
> > +{
> > + struct bcm2835_isp_node *node = video_drvdata(file);
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct vchiq_mmal_port *port = get_port_data(node);
> > + struct mmal_parameter_crop crop;
> > + u32 crop_size = sizeof(crop);
> > + int ret;
> > +
> > + /* This return value is required for V4L2 compliance. */
> > + if (node_is_stats(node))
> > + return -ENOTTY;
> > +
> > + /* We can only return out an input crop. */
> > + if (s->target != V4L2_SEL_TGT_CROP)
> > + return -EINVAL;
>
> No support for _TGT_CROP_DEFAULT/BOUNDS? Those are usually supported
> by drivers and are typically set to the width/height of the current
> format.
>
> I recommend adding support for these targets.
Ack
>
> > +
> > + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance, port,
> > + MMAL_PARAMETER_CROP,
> > + &crop, &crop_size);
> > + if (!ret)
> > + return -EINVAL;
> > +
> > + s->r.left = crop.rect.x;
> > + s->r.top = crop.rect.y;
> > + s->r.width = crop.rect.width;
> > + s->r.height = crop.rect.height;
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm3285_isp_subscribe_event(struct v4l2_fh *fh,
> > + const struct v4l2_event_subscription *s)
> > +{
> > + switch (s->type) {
> > + /* Cannot change source parameters dynamically at runtime. */
> > + case V4L2_EVENT_SOURCE_CHANGE:
> > + return -EINVAL;
> > + case V4L2_EVENT_CTRL:
> > + return v4l2_ctrl_subscribe_event(fh, s);
> > + default:
> > + return v4l2_event_subscribe(fh, s, 4, NULL);
> > + }
> > +}
> > +
> > +static const struct v4l2_ioctl_ops bcm2835_isp_node_ioctl_ops = {
> > + .vidioc_querycap = bcm2835_isp_node_querycap,
> > + .vidioc_g_fmt_vid_cap = bcm2835_isp_node_g_fmt,
> > + .vidioc_g_fmt_vid_out = bcm2835_isp_node_g_fmt,
> > + .vidioc_g_fmt_meta_cap = bcm2835_isp_node_g_fmt,
> > + .vidioc_s_fmt_vid_cap = bcm2835_isp_node_s_fmt,
> > + .vidioc_s_fmt_vid_out = bcm2835_isp_node_s_fmt,
> > + .vidioc_s_fmt_meta_cap = bcm2835_isp_node_s_fmt,
> > + .vidioc_try_fmt_vid_cap = bcm2835_isp_node_try_fmt,
> > + .vidioc_try_fmt_vid_out = bcm2835_isp_node_try_fmt,
> > + .vidioc_try_fmt_meta_cap = bcm2835_isp_node_try_fmt,
> > + .vidioc_s_selection = bcm2835_isp_node_s_selection,
> > + .vidioc_g_selection = bcm2835_isp_node_g_selection,
> > +
> > + .vidioc_enum_fmt_vid_cap = bcm2835_isp_node_enum_fmt,
> > + .vidioc_enum_fmt_vid_out = bcm2835_isp_node_enum_fmt,
> > + .vidioc_enum_fmt_meta_cap = bcm2835_isp_node_enum_fmt,
> > + .vidioc_enum_framesizes = bcm2835_isp_enum_framesizes,
> > +
> > + .vidioc_reqbufs = vb2_ioctl_reqbufs,
> > + .vidioc_querybuf = vb2_ioctl_querybuf,
> > + .vidioc_qbuf = vb2_ioctl_qbuf,
> > + .vidioc_dqbuf = vb2_ioctl_dqbuf,
> > + .vidioc_expbuf = vb2_ioctl_expbuf,
> > + .vidioc_create_bufs = vb2_ioctl_create_bufs,
> > + .vidioc_prepare_buf = vb2_ioctl_prepare_buf,
> > +
> > + .vidioc_streamon = vb2_ioctl_streamon,
> > + .vidioc_streamoff = vb2_ioctl_streamoff,
> > +
> > + .vidioc_subscribe_event = bcm3285_isp_subscribe_event,
> > + .vidioc_unsubscribe_event = v4l2_event_unsubscribe,
> > +};
> > +
> > +/*
> > + * Size of the array to provide to the VPU when asking for the list of supported
> > + * formats.
> > + *
> > + * The ISP component currently advertises 33 input formats, so add a small
> > + * overhead on that.
> > + */
> > +#define MAX_SUPPORTED_ENCODINGS 40
> > +
> > +/* Populate node->supported_fmts with the formats supported by those ports. */
> > +static int bcm2835_isp_get_supported_fmts(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > + struct bcm2835_isp_fmt const **list;
> > + unsigned int i, j, num_encodings;
> > + u32 fourccs[MAX_SUPPORTED_ENCODINGS];
> > + u32 param_size = sizeof(fourccs);
> > + int ret;
> > +
> > + ret = vchiq_mmal_port_parameter_get(dev->mmal_instance,
> > + get_port_data(node),
> > + MMAL_PARAMETER_SUPPORTED_ENCODINGS,
> > + &fourccs, ¶m_size);
> > +
> > + if (ret) {
> > + if (ret == MMAL_MSG_STATUS_ENOSPC) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: port has more encoding than we provided space for. Some are dropped.\n",
> > + __func__);
> > + num_encodings = MAX_SUPPORTED_ENCODINGS;
> > + } else {
> > + v4l2_err(&dev->v4l2_dev, "%s: get_param ret %u.\n",
> > + __func__, ret);
> > + return -EINVAL;
> > + }
> > + } else {
> > + num_encodings = param_size / sizeof(u32);
> > + }
> > +
> > + /*
> > + * Assume at this stage that all encodings will be supported in V4L2.
> > + * Any that aren't supported will waste a very small amount of memory.
> > + */
> > + list = devm_kzalloc(dev->dev,
> > + sizeof(struct bcm2835_isp_fmt *) * num_encodings,
> > + GFP_KERNEL);
> > + if (!list)
> > + return -ENOMEM;
> > + node->supported_fmts.list = list;
> > +
> > + for (i = 0, j = 0; i < num_encodings; i++) {
> > + const struct bcm2835_isp_fmt *fmt = get_fmt(fourccs[i]);
> > +
> > + if (fmt) {
> > + list[j] = fmt;
> > + j++;
> > + }
> > + }
> > + node->supported_fmts.num_entries = j;
> > +
> > + return 0;
> > +}
> > +
> > +/*
> > + * Register a device node /dev/video<N> to go along with one of the ISP's input
> > + * or output nodes.
> > + */
> > +static int register_node(struct bcm2835_isp_dev *dev,
> > + struct bcm2835_isp_node *node,
> > + int index)
> > +{
> > + struct video_device *vfd;
> > + struct vb2_queue *queue;
> > + int ret;
> > +
> > + mutex_init(&node->lock);
> > +
> > + node->dev = dev;
> > + vfd = &node->vfd;
> > + queue = &node->queue;
> > + queue->type = index_to_queue_type(index);
> > + /*
> > + * Setup the node type-specific params.
> > + *
> > + * Only the OUTPUT node can set controls and crop windows. However,
> > + * we must allow the s/g_selection ioctl on the stats node as v4l2
> > + * compliance expects it to return a -ENOTTY, and the framework
> > + * does not handle it if the ioctl is disabled.
> > + */
> > + switch (queue->type) {
> > + case V4L2_BUF_TYPE_VIDEO_OUTPUT:
> > + vfd->device_caps = V4L2_CAP_VIDEO_OUTPUT | V4L2_CAP_STREAMING;
> > + node->id = index;
> > + node->vfl_dir = VFL_DIR_TX;
> > + node->name = "output";
> > + break;
> > + case V4L2_BUF_TYPE_VIDEO_CAPTURE:
> > + vfd->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_STREAMING;
> > + /* First Capture node starts at id 0, etc. */
> > + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
> > + node->vfl_dir = VFL_DIR_RX;
> > + node->name = "capture";
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_SELECTION);
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_G_SELECTION);
> > + break;
> > + case V4L2_BUF_TYPE_META_CAPTURE:
> > + vfd->device_caps = V4L2_CAP_META_CAPTURE | V4L2_CAP_STREAMING;
> > + node->id = index - BCM2835_ISP_NUM_OUTPUTS;
> > + node->vfl_dir = VFL_DIR_RX;
> > + node->name = "stats";
> > + v4l2_disable_ioctl(&node->vfd, VIDIOC_S_CTRL);
>
> Why not disable S/G_SELECTION for meta capture here rather than test for it
> in the op functions?
The impelmenation with S/G_SELECTION for meta capture disabled caused
problem with v4l2-compliance. However, with the other changes
proposed in this review, it does seem to now work correcly. So I will
update this code and disable the ioctl for the meta capture node.
>
> > + break;
> > + }
> > +
> > + /* We use the selection API instead of the old crop API. */
> > + v4l2_disable_ioctl(vfd, VIDIOC_CROPCAP);
> > + v4l2_disable_ioctl(vfd, VIDIOC_G_CROP);
> > + v4l2_disable_ioctl(vfd, VIDIOC_S_CROP);
>
> No need for this: the core handles this and will disable these ioctls
> automatically.
Without these explicitally disabled, we fail v4l2-compliance on the
video capture nodes with the following error:
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
test VIDIOC_G/S_PARM: OK (Not Supported)
test VIDIOC_G_FBUF: OK (Not Supported)
test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
fail: v4l2-test-formats.cpp(1510): doioctl(node, VIDIOC_G_SELECTION, &sel)
fail: v4l2-test-formats.cpp(1550): testLegacyCrop(node)
test Cropping: FAIL
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
>
> > +
> > + ret = bcm2835_isp_get_supported_fmts(node);
> > + if (ret)
> > + return ret;
> > +
> > + /* Initialise the the video node. */
> > + vfd->vfl_type = VFL_TYPE_VIDEO;
> > + vfd->fops = &bcm2835_isp_fops,
> > + vfd->ioctl_ops = &bcm2835_isp_node_ioctl_ops,
> > + vfd->minor = -1,
> > + vfd->release = video_device_release_empty,
> > + vfd->queue = &node->queue;
> > + vfd->lock = &node->lock;
> > + vfd->v4l2_dev = &dev->v4l2_dev;
> > + vfd->vfl_dir = node->vfl_dir;
> > +
> > + node->q_data.fmt = get_default_format(node);
> > + node->q_data.width = DEFAULT_DIM;
> > + node->q_data.height = DEFAULT_DIM;
> > + node->q_data.bytesperline =
> > + get_bytesperline(DEFAULT_DIM, node->q_data.fmt);
> > + node->q_data.sizeimage = node_is_stats(node) ?
> > + get_port_data(node)->recommended_buffer.size :
> > + get_sizeimage(node->q_data.bytesperline,
> > + node->q_data.width,
> > + node->q_data.height,
> > + node->q_data.fmt);
> > +
> > + queue->io_modes = VB2_MMAP | VB2_DMABUF;
> > + queue->drv_priv = node;
> > + queue->ops = &bcm2835_isp_node_queue_ops;
> > + queue->mem_ops = &vb2_dma_contig_memops;
> > + queue->buf_struct_size = sizeof(struct bcm2835_isp_buffer);
> > + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
> > + queue->dev = dev->dev;
> > + queue->lock = &node->queue_lock;
> > +
> > + ret = vb2_queue_init(queue);
> > + if (ret < 0) {
> > + v4l2_info(&dev->v4l2_dev, "vb2_queue_init failed\n");
> > + return ret;
> > + }
> > + node->queue_init = true;
> > +
> > + /* Define the device names */
> > + snprintf(vfd->name, sizeof(node->vfd.name), "%s-%s%d", BCM2835_ISP_NAME,
> > + node->name, node->id);
> > +
> > + ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr + index);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "Failed to register video %s[%d] device node\n",
> > + node->name, node->id);
> > + return ret;
> > + }
>
> Move registering the video device to the end of this function.
> Otherwise the output video device would be created (and available for
> userspace) before the controls are added.
>
Ack
> > +
> > + node->registered = true;
> > + video_set_drvdata(vfd, node);
> > +
> > + /* Set some controls and defaults, but only on the VIDEO_OUTPUT node. */
> > + if (node_is_output(node)) {
> > + unsigned int i;
> > +
> > + /* Use this ctrl template to assign all out ISP custom ctrls. */
> > + struct v4l2_ctrl_config ctrl_template = {
> > + .ops = &bcm2835_isp_ctrl_ops,
> > + .type = V4L2_CTRL_TYPE_U8,
> > + .def = 0,
> > + .min = 0x00,
> > + .max = 0xff,
> > + .step = 1,
> > + };
> > +
> > + v4l2_ctrl_handler_init(&dev->ctrl_handler, 4);
> > +
> > + dev->r_gain = 1000;
> > + dev->b_gain = 1000;
> > +
> > + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> > + V4L2_CID_RED_BALANCE, 1, 0xffff, 1,
> > + dev->r_gain);
> > +
> > + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> > + V4L2_CID_BLUE_BALANCE, 1, 0xffff, 1,
> > + dev->b_gain);
> > +
> > + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
> > + V4L2_CID_DIGITAL_GAIN, 1, 0xffff, 1, 1000);
> > +
> > + for (i = 0; i < ARRAY_SIZE(custom_ctrls); i++) {
> > + ctrl_template.name = custom_ctrls[i].name;
> > + ctrl_template.id = custom_ctrls[i].id;
> > + ctrl_template.dims[0] = custom_ctrls[i].size;
> > + ctrl_template.flags = custom_ctrls[i].flags;
> > + v4l2_ctrl_new_custom(&dev->ctrl_handler,
> > + &ctrl_template, NULL);
> > + }
> > +
> > + node->vfd.ctrl_handler = &dev->ctrl_handler;
>
> Missing error check.
Ack
>
> > + }
> > +
> > + v4l2_info(&dev->v4l2_dev,
> > + "Device node %s[%d] registered as /dev/video%d\n",
> > + node->name, node->id, vfd->num);
> > +
> > + return 0;
> > +}
> > +
> > +/* Unregister one of the /dev/video<N> nodes associated with the ISP. */
> > +static void unregister_node(struct bcm2835_isp_node *node)
> > +{
> > + struct bcm2835_isp_dev *dev = node_get_dev(node);
> > +
> > + v4l2_info(&dev->v4l2_dev,
> > + "Unregistering node %s[%d] device node /dev/video%d\n",
> > + node->name, node->id, node->vfd.num);
> > +
> > + if (node->queue_init)
> > + vb2_queue_release(&node->queue);
> > +
> > + if (node->registered) {
> > + video_unregister_device(&node->vfd);
> > + if (node_is_output(node))
> > + v4l2_ctrl_handler_free(&dev->ctrl_handler);
> > + }
> > +
> > + /*
> > + * node->supported_fmts.list is free'd automatically
> > + * as a managed resource.
> > + */
> > + node->supported_fmts.list = NULL;
> > + node->supported_fmts.num_entries = 0;
> > + node->vfd.ctrl_handler = NULL;
> > + node->registered = false;
> > + node->queue_init = false;
> > +}
> > +
> > +static void media_controller_unregister(struct bcm2835_isp_dev *dev)
> > +{
> > + unsigned int i;
> > +
> > + v4l2_info(&dev->v4l2_dev, "Unregister from media controller\n");
> > +
> > + if (dev->media_device_registered) {
> > + media_device_unregister(&dev->mdev);
> > + media_device_cleanup(&dev->mdev);
> > + dev->media_device_registered = false;
> > + }
> > +
> > + kfree(dev->entity.name);
> > + dev->entity.name = NULL;
> > +
> > + if (dev->media_entity_registered) {
> > + media_device_unregister_entity(&dev->entity);
> > + dev->media_entity_registered = false;
> > + }
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + struct bcm2835_isp_node *node = &dev->node[i];
> > +
> > + if (node->media_node_registered) {
> > + media_remove_intf_links(node->intf_link->intf);
> > + media_entity_remove_links(&dev->node[i].vfd.entity);
> > + media_devnode_remove(node->intf_devnode);
> > + media_device_unregister_entity(&node->vfd.entity);
> > + kfree(node->vfd.entity.name);
> > + }
> > + node->media_node_registered = false;
> > + }
> > +
> > + dev->v4l2_dev.mdev = NULL;
> > +}
> > +
> > +static int media_controller_register_node(struct bcm2835_isp_dev *dev, int num)
> > +{
> > + struct bcm2835_isp_node *node = &dev->node[num];
> > + struct media_entity *entity = &node->vfd.entity;
> > + int output = node_is_output(node);
> > + char *name;
> > + int ret;
> > +
> > + v4l2_info(&dev->v4l2_dev,
> > + "Register %s node %d with media controller\n",
> > + output ? "output" : "capture", num);
> > + entity->obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
> > + entity->function = MEDIA_ENT_F_IO_V4L;
> > + entity->info.dev.major = VIDEO_MAJOR;
> > + entity->info.dev.minor = node->vfd.minor;
> > + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
> > + if (!name) {
> > + ret = -ENOMEM;
> > + goto error_no_mem;
> > + }
> > + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "%s0-%s%d",
> > + BCM2835_ISP_NAME, output ? "output" : "capture", num);
> > + entity->name = name;
> > + node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
> > + ret = media_entity_pads_init(entity, 1, &node->pad);
> > + if (ret)
> > + goto error_pads_init;
> > + ret = media_device_register_entity(&dev->mdev, entity);
> > + if (ret)
> > + goto error_register_entity;
> > +
> > + node->intf_devnode = media_devnode_create(&dev->mdev,
> > + MEDIA_INTF_T_V4L_VIDEO, 0,
> > + VIDEO_MAJOR, node->vfd.minor);
> > + if (!node->intf_devnode) {
> > + ret = -ENOMEM;
> > + goto error_devnode_create;
> > + }
> > +
> > + node->intf_link = media_create_intf_link(entity,
> > + &node->intf_devnode->intf,
> > + MEDIA_LNK_FL_IMMUTABLE |
> > + MEDIA_LNK_FL_ENABLED);
> > + if (!node->intf_link) {
> > + ret = -ENOMEM;
> > + goto error_create_intf_link;
> > + }
> > +
> > + if (output)
> > + ret = media_create_pad_link(entity, 0, &dev->entity, num,
> > + MEDIA_LNK_FL_IMMUTABLE |
> > + MEDIA_LNK_FL_ENABLED);
> > + else
> > + ret = media_create_pad_link(&dev->entity, num, entity, 0,
> > + MEDIA_LNK_FL_IMMUTABLE |
> > + MEDIA_LNK_FL_ENABLED);
> > + if (ret)
> > + goto error_create_pad_link;
> > +
> > + dev->node[num].media_node_registered = true;
> > + return 0;
> > +
> > +error_create_pad_link:
> > + media_remove_intf_links(&node->intf_devnode->intf);
> > +error_create_intf_link:
> > + media_devnode_remove(node->intf_devnode);
> > +error_devnode_create:
> > + media_device_unregister_entity(&node->vfd.entity);
> > +error_register_entity:
> > +error_pads_init:
> > + kfree(entity->name);
> > + entity->name = NULL;
> > +error_no_mem:
> > + if (ret)
> > + v4l2_info(&dev->v4l2_dev, "Error registering node\n");
> > +
> > + return ret;
> > +}
> > +
> > +static int media_controller_register(struct bcm2835_isp_dev *dev)
> > +{
> > + char *name;
> > + unsigned int i;
> > + int ret;
> > +
> > + v4l2_dbg(2, debug, &dev->v4l2_dev, "Registering with media controller\n");
> > + dev->mdev.dev = dev->dev;
> > + strscpy(dev->mdev.model, "bcm2835-isp",
> > + sizeof(dev->mdev.model));
> > + strscpy(dev->mdev.bus_info, "platform:bcm2835-isp",
> > + sizeof(dev->mdev.bus_info));
> > + media_device_init(&dev->mdev);
> > + dev->v4l2_dev.mdev = &dev->mdev;
> > +
> > + v4l2_dbg(2, debug, &dev->v4l2_dev, "Register entity for nodes\n");
> > +
> > + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
> > + if (!name) {
> > + ret = -ENOMEM;
> > + goto done;
> > + }
> > + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "bcm2835_isp0");
> > + dev->entity.name = name;
> > + dev->entity.obj_type = MEDIA_ENTITY_TYPE_BASE;
> > + dev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + dev->pad[i].flags = node_is_output(&dev->node[i]) ?
> > + MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
> > + }
> > +
> > + ret = media_entity_pads_init(&dev->entity, BCM2835_ISP_NUM_NODES,
> > + dev->pad);
> > + if (ret)
> > + goto done;
> > +
> > + ret = media_device_register_entity(&dev->mdev, &dev->entity);
> > + if (ret)
> > + goto done;
> > +
> > + dev->media_entity_registered = true;
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + ret = media_controller_register_node(dev, i);
> > + if (ret)
> > + goto done;
> > + }
> > +
> > + ret = media_device_register(&dev->mdev);
> > + if (!ret)
> > + dev->media_device_registered = true;
> > +done:
> > + return ret;
> > +}
> > +
> > +static int bcm2835_isp_remove(struct platform_device *pdev)
> > +{
> > + struct bcm2835_isp_dev *dev = platform_get_drvdata(pdev);
> > + unsigned int i;
> > +
> > + media_controller_unregister(dev);
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++)
> > + unregister_node(&dev->node[i]);
> > +
> > + v4l2_device_unregister(&dev->v4l2_dev);
> > +
> > + if (dev->component)
> > + vchiq_mmal_component_finalise(dev->mmal_instance,
> > + dev->component);
> > +
> > + vchiq_mmal_finalise(dev->mmal_instance);
> > +
> > + return 0;
> > +}
> > +
> > +static int bcm2835_isp_probe(struct platform_device *pdev)
> > +{
> > + struct bcm2835_isp_dev *dev;
> > + unsigned int i;
> > + int ret;
> > +
> > + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
> > + if (!dev)
> > + return -ENOMEM;
> > +
> > + dev->dev = &pdev->dev;
> > +
> > + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
> > + if (ret)
> > + return ret;
> > +
> > + ret = vchiq_mmal_init(&dev->mmal_instance);
> > + if (ret) {
> > + v4l2_device_unregister(&dev->v4l2_dev);
> > + return ret;
> > + }
> > +
> > + ret = vchiq_mmal_component_init(dev->mmal_instance, "ril.isp",
> > + &dev->component);
> > + if (ret) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: failed to create ril.isp component\n", __func__);
> > + goto error;
> > + }
> > +
> > + if ((dev->component->inputs != BCM2835_ISP_NUM_OUTPUTS) ||
> > + (dev->component->outputs != BCM2835_ISP_NUM_CAPTURES +
> > + BCM2835_ISP_NUM_METADATA)) {
> > + v4l2_err(&dev->v4l2_dev,
> > + "%s: ril.isp returned %d i/p (%d expected), %d o/p (%d expected) ports\n",
> > + __func__, dev->component->inputs,
> > + BCM2835_ISP_NUM_OUTPUTS,
> > + dev->component->outputs,
> > + BCM2835_ISP_NUM_CAPTURES + BCM2835_ISP_NUM_METADATA);
> > + goto error;
> > + }
> > +
> > + atomic_set(&dev->num_streaming, 0);
> > +
> > + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
> > + struct bcm2835_isp_node *node = &dev->node[i];
> > +
> > + ret = register_node(dev, node, i);
> > + if (ret)
> > + goto error;
> > + }
> > +
> > + ret = media_controller_register(dev);
> > + if (ret)
> > + goto error;
> > +
> > + platform_set_drvdata(pdev, dev);
> > + v4l2_info(&dev->v4l2_dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
> > + return 0;
> > +
> > +error:
> > + bcm2835_isp_remove(pdev);
> > +
> > + return ret;
> > +}
> > +
> > +static struct platform_driver bcm2835_isp_pdrv = {
> > + .probe = bcm2835_isp_probe,
> > + .remove = bcm2835_isp_remove,
> > + .driver = {
> > + .name = BCM2835_ISP_NAME,
> > + },
> > +};
> > +
> > +module_platform_driver(bcm2835_isp_pdrv);
> > +
> > +MODULE_DESCRIPTION("BCM2835 ISP driver");
> > +MODULE_AUTHOR("Naushir Patuck <naush@raspberrypi.com>");
> > +MODULE_LICENSE("GPL");
> > +MODULE_VERSION("1.0");
> > +MODULE_ALIAS("platform:bcm2835-isp");
> > diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> > new file mode 100644
> > index 000000000000..cfbb1063aad1
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
> > @@ -0,0 +1,67 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Broadcom BCM2835 ISP driver
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#ifndef BCM2835_ISP_CTRLS
> > +#define BCM2835_ISP_CTRLS
> > +
> > +#include <linux/bcm2835-isp.h>
> > +
> > +struct bcm2835_isp_custom_ctrl {
> > + const char *name;
> > + u32 id;
> > + u32 size;
> > + u32 flags;
> > +};
> > +
> > +static const struct bcm2835_isp_custom_ctrl custom_ctrls[] = {
> > + {
> > + .name = "Colour Correction Matrix",
> > + .id = V4L2_CID_USER_BCM2835_ISP_CC_MATRIX,
> > + .size = sizeof(struct bcm2835_isp_custom_ccm),
> > + .flags = 0
> > + }, {
> > + .name = "Lens Shading",
> > + .id = V4L2_CID_USER_BCM2835_ISP_LENS_SHADING,
> > + .size = sizeof(struct bcm2835_isp_lens_shading),
> > + .flags = V4L2_CTRL_FLAG_EXECUTE_ON_WRITE
> > + }, {
> > + .name = "Black Level",
> > + .id = V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL,
> > + .size = sizeof(struct bcm2835_isp_black_level),
> > + .flags = 0
> > + }, {
> > + .name = "Green Equalisation",
> > + .id = V4L2_CID_USER_BCM2835_ISP_GEQ,
> > + .size = sizeof(struct bcm2835_isp_geq),
> > + .flags = 0
> > + }, {
> > + .name = "Gamma",
> > + .id = V4L2_CID_USER_BCM2835_ISP_GAMMA,
> > + .size = sizeof(struct bcm2835_isp_gamma),
> > + .flags = 0
> > + }, {
> > + .name = "Sharpen",
> > + .id = V4L2_CID_USER_BCM2835_ISP_SHARPEN,
> > + .size = sizeof(struct bcm2835_isp_sharpen),
> > + .flags = 0
> > + }, {
> > + .name = "Denoise",
> > + .id = V4L2_CID_USER_BCM2835_ISP_DENOISE,
> > + .size = sizeof(struct bcm2835_isp_denoise),
> > + .flags = 0
> > + }, {
> > + .name = "Defective Pixel Correction",
> > + .id = V4L2_CID_USER_BCM2835_ISP_DPC,
> > + .size = sizeof(struct bcm2835_isp_dpc),
> > + .flags = 0
> > + }
> > +};
> > +
> > +#endif
> > diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> > new file mode 100644
> > index 000000000000..af3bde152bb2
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
> > @@ -0,0 +1,301 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +/*
> > + * Broadcom BCM2835 ISP driver
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#ifndef BCM2835_ISP_FMTS
> > +#define BCM2835_ISP_FMTS
> > +
> > +#include <linux/videodev2.h>
> > +#include "vchiq-mmal/mmal-encodings.h"
> > +
> > +struct bcm2835_isp_fmt {
> > + u32 fourcc;
> > + int depth;
> > + int bytesperline_align;
> > + u32 flags;
> > + u32 mmal_fmt;
> > + int size_multiplier_x2;
> > + enum v4l2_colorspace colorspace;
> > + unsigned int step_size;
> > +};
> > +
> > +struct bcm2835_isp_fmt_list {
> > + struct bcm2835_isp_fmt const **list;
> > + unsigned int num_entries;
> > +};
> > +
> > +static const struct bcm2835_isp_fmt supported_formats[] = {
> > + {
> > + /* YUV formats */
> > + .fourcc = V4L2_PIX_FMT_YUV420,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_I420,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YVU420,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_YV12,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_NV12,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_NV12,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_NV21,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_NV21,
> > + .size_multiplier_x2 = 3,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YUYV,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_YUYV,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_UYVY,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_UYVY,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_YVYU,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_YVYU,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_VYUY,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_VYUY,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SMPTE170M,
> > + .step_size = 2,
> > + }, {
> > + /* RGB formats */
> > + .fourcc = V4L2_PIX_FMT_RGB24,
> > + .depth = 24,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_RGB24,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_RGB565,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_RGB16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_BGR24,
> > + .depth = 24,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BGR24,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_ABGR32,
> > + .depth = 32,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BGRA,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_SRGB,
> > + .step_size = 1,
> > + }, {
> > + /* Bayer formats */
> > + /* 8 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG8,
> > + .depth = 8,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG8,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* 10 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG10P,
> > + .depth = 10,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* 12 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG12P,
> > + .depth = 12,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12P,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* 16 bit */
> > + .fourcc = V4L2_PIX_FMT_SRGGB16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SBGGR16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGRBG16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + .fourcc = V4L2_PIX_FMT_SGBRG16,
> > + .depth = 16,
> > + .bytesperline_align = 32,
> > + .flags = 0,
> > + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG16,
> > + .size_multiplier_x2 = 2,
> > + .colorspace = V4L2_COLORSPACE_RAW,
> > + .step_size = 2,
> > + }, {
> > + /* ISP statistics format */
> > + .fourcc = V4L2_META_FMT_BCM2835_ISP_STATS,
> > + .mmal_fmt = MMAL_ENCODING_BRCM_STATS,
> > + /* The rest are not valid fields for stats. */
> > + }
> > +};
> > +
> > +#endif
> > diff --git a/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> > new file mode 100644
> > index 000000000000..edc452fa8318
> > --- /dev/null
> > +++ b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> > @@ -0,0 +1,333 @@
> > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> > +/*
> > + * bcm2835-isp.h
> > + *
> > + * BCM2835 ISP driver - user space header file.
> > + *
> > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > + *
> > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > + *
> > + */
> > +
> > +#ifndef __BCM2835_ISP_H_
> > +#define __BCM2835_ISP_H_
> > +
> > +#include <linux/v4l2-controls.h>
> > +
> > +/* TODO: move the control IDs definitions to v4l2-controls.h */
> > +#define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10c0)
>
> As the TODO says: move this to v4l2-controls.h. Currently the 0x10c0 offset
> clashes with V4L2_CID_USER_ATMEL_ISC_BASE, so that certainly should be fixed.
>
Unfortunately, there seems to be a mixup here. Laurent, we have
accidentally mailed a WIP revision of this patch. The final version
does have V4L2_CID_USER_BCM2835_ISP_BASE with a unique id in
v4l2-controls.h. I will talk with Laurent separately to get the
correct revison included in the next patch-set.
> > +
> > +/* TODO: move the formats definitions to videodev2.h */
> > +/* 12 Y/CbCr 4:2:0 128 pixel wide column */
> > +#define V4L2_PIX_FMT_NV12_COL128 v4l2_fourcc('N', 'C', '1', '2')
> > +/* Y/CbCr 4:2:0 10bpc, 3x10 packed as 4 bytes in a 128 bytes / 96 pixel wide column */
> > +#define V4L2_PIX_FMT_NV12_10_COL128 v4l2_fourcc('N', 'C', '3', '0')
> > +/* Sensor Ancillary metadata */
> > +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S')
> > +/* BCM2835 ISP image statistics output */
> > +#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A')
> > +
Similarly, these have also been moved to the right header files.
> > +#define V4L2_CID_USER_BCM2835_ISP_CC_MATRIX \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0001)
> > +#define V4L2_CID_USER_BCM2835_ISP_LENS_SHADING \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0002)
> > +#define V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0003)
> > +#define V4L2_CID_USER_BCM2835_ISP_GEQ \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0004)
> > +#define V4L2_CID_USER_BCM2835_ISP_GAMMA \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0005)
> > +#define V4L2_CID_USER_BCM2835_ISP_DENOISE \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0006)
> > +#define V4L2_CID_USER_BCM2835_ISP_SHARPEN \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
> > +#define V4L2_CID_USER_BCM2835_ISP_DPC \
> > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
>
> There is no documentation for these controls. Specifically, it doesn't
> tell you which struct should be used.
As above, the documentaiton is available in the newer patch.
>
> > +
> > +/*
> > + * All structs below are directly mapped onto the equivalent structs in
> > + * drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > + * for convenience.
> > + */
> > +
> > +/**
> > + * struct bcm2835_isp_rational - Rational value type.
> > + *
> > + * @num: Numerator.
> > + * @den: Denominator.
> > + */
> > +struct bcm2835_isp_rational {
> > + __s32 num;
> > + __s32 den;
>
> Wouldn't it make more sense if den is a __u32?
Ack
>
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_ccm - Colour correction matrix.
> > + *
> > + * @ccm: 3x3 correction matrix coefficients.
> > + * @offsets: 1x3 correction offsets.
> > + */
> > +struct bcm2835_isp_ccm {
> > + struct bcm2835_isp_rational ccm[3][3];
> > + __s32 offsets[3];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_custom_ccm - Custom CCM applied with the
> > + * V4L2_CID_USER_BCM2835_ISP_CC_MATRIX ctrl.
> > + *
> > + * @enabled: Enable custom CCM.
> > + * @ccm: Custom CCM coefficients and offsets.
> > + */
> > +struct bcm2835_isp_custom_ccm {
> > + __u32 enabled;
> > + struct bcm2835_isp_ccm ccm;
> > +};
> > +
> > +/**
> > + * enum bcm2835_isp_gain_format - format of the gains in the lens shading
> > + * tables used with the
> > + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING ctrl.
> > + *
> > + * @GAIN_FORMAT_U0P8_1: Gains are u0.8 format, starting at 1.0
> > + * @GAIN_FORMAT_U1P7_0: Gains are u1.7 format, starting at 0.0
> > + * @GAIN_FORMAT_U1P7_1: Gains are u1.7 format, starting at 1.0
> > + * @GAIN_FORMAT_U2P6_0: Gains are u2.6 format, starting at 0.0
> > + * @GAIN_FORMAT_U2P6_1: Gains are u2.6 format, starting at 1.0
> > + * @GAIN_FORMAT_U3P5_0: Gains are u3.5 format, starting at 0.0
> > + * @GAIN_FORMAT_U3P5_1: Gains are u3.5 format, starting at 1.0
> > + * @GAIN_FORMAT_U4P10: Gains are u4.10 format, starting at 0.0
> > + */
> > +enum bcm2835_isp_gain_format {
> > + GAIN_FORMAT_U0P8_1 = 0,
> > + GAIN_FORMAT_U1P7_0 = 1,
> > + GAIN_FORMAT_U1P7_1 = 2,
> > + GAIN_FORMAT_U2P6_0 = 3,
> > + GAIN_FORMAT_U2P6_1 = 4,
> > + GAIN_FORMAT_U3P5_0 = 5,
> > + GAIN_FORMAT_U3P5_1 = 6,
> > + GAIN_FORMAT_U4P10 = 7,
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_lens_shading - Lens shading tables supplied with the
> > + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING
> > + * ctrl.
> > + *
> > + * @enabled: Enable lens shading.
> > + * @grid_cell_size: Size of grid cells in samples (16, 32, 64, 128 or 256).
> > + * @grid_width: Width of lens shading tables in grid cells.
> > + * @grid_stride: Row to row distance (in grid cells) between grid cells
> > + * in the same horizontal location.
> > + * @grid_height: Height of lens shading tables in grid cells.
> > + * @mem_handle_table: Memory handle to the tables.
>
> What sort of handle is this? I.e. where does it come from?
I believe there was a separte discusion about this. This is a vcsm
handle that is used to store the table coefficients in memory that is
accessible from Videocore. There is work in progress to update this
to use dmabuf handles.
>
> > + * @ref_transform: Reference transform - unsupported, please pass zero.
> > + * @corner_sampled: Whether the gains are sampled at the corner points
> > + * of the grid cells or in the cell centres.
> > + * @gain_format: Format of the gains (see enum &bcm2835_isp_gain_format).
> > + */
> > +struct bcm2835_isp_lens_shading {
> > + __u32 enabled;
> > + __u32 grid_cell_size;
> > + __u32 grid_width;
> > + __u32 grid_stride;
> > + __u32 grid_height;
> > + __u32 mem_handle_table;
> > + __u32 ref_transform;
> > + __u32 corner_sampled;
> > + __u32 gain_format;
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_black_level - Sensor black level set with the
> > + * V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL ctrl.
> > + *
> > + * @enabled: Enable black level.
> > + * @black_level_r: Black level for red channel.
> > + * @black_level_g: Black level for green channels.
> > + * @black_level_b: Black level for blue channel.
> > + */
> > +struct bcm2835_isp_black_level {
> > + __u32 enabled;
> > + __u16 black_level_r;
> > + __u16 black_level_g;
> > + __u16 black_level_b;
> > + __u8 pad_[2]; /* Unused */
>
> I prefer 'padding' over 'pad_'.
Ack
>
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_geq - Green equalisation parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_GEQ ctrl.
> > + *
> > + * @enabled: Enable green equalisation.
> > + * @offset: Fixed offset of the green equalisation threshold.
> > + * @slope: Slope of the green equalisation threshold.
> > + */
> > +struct bcm2835_isp_geq {
> > + __u32 enabled;
> > + __u32 offset;
> > + struct bcm2835_isp_rational slope;
> > +};
> > +
> > +#define BCM2835_NUM_GAMMA_PTS 33
> > +
> > +/**
> > + * struct bcm2835_isp_gamma - Gamma parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_GAMMA ctrl.
> > + *
> > + * @enabled: Enable gamma adjustment.
> > + * @X: X values of the points defining the gamma curve.
> > + * Values should be scaled to 16 bits.
> > + * @Y: Y values of the points defining the gamma curve.
> > + * Values should be scaled to 16 bits.
>
> I assume 0 == black and 0xffff == white (or max luminance)?
>
> And so typically x[0] == y[0] == 0 and x[32] == y[32] == 0xffff?
>
Typically yes, but it is not strictily true. We could have parameters
that clip the signal above min (0) and below max (0xffff).
> > + */
> > +struct bcm2835_isp_gamma {
> > + __u32 enabled;
> > + __u16 x[BCM2835_NUM_GAMMA_PTS];
> > + __u16 y[BCM2835_NUM_GAMMA_PTS];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_denoise - Denoise parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_DENOISE ctrl.
> > + *
> > + * @enabled: Enable denoise.
> > + * @constant: Fixed offset of the noise threshold.
> > + * @slope: Slope of the noise threshold.
> > + * @strength: Denoise strength between 0.0 (off) and 1.0 (maximum).
> > + */
> > +struct bcm2835_isp_denoise {
> > + __u32 enabled;
> > + __u32 constant;
> > + struct bcm2835_isp_rational slope;
> > + struct bcm2835_isp_rational strength;
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_sharpen - Sharpen parameters set with the
> > + * V4L2_CID_USER_BCM2835_ISP_SHARPEN ctrl.
> > + *
> > + * @enabled: Enable sharpening.
> > + * @threshold: Threshold at which to start sharpening pixels.
> > + * @strength: Strength with which pixel sharpening increases.
> > + * @limit: Limit to the amount of sharpening applied.
> > + */
> > +struct bcm2835_isp_sharpen {
> > + __u32 enabled;
> > + struct bcm2835_isp_rational threshold;
> > + struct bcm2835_isp_rational strength;
> > + struct bcm2835_isp_rational limit;
> > +};
> > +
> > +/**
> > + * enum bcm2835_isp_dpc_mode - defective pixel correction (DPC) strength.
> > + *
> > + * @DPC_MODE_OFF: No DPC.
> > + * @DPC_MODE_NORMAL: Normal DPC.
> > + * @DPC_MODE_STRONG: Strong DPC.
> > + */
> > +enum bcm2835_isp_dpc_mode {
> > + DPC_MODE_OFF = 0,
> > + DPC_MODE_NORMAL = 1,
> > + DPC_MODE_STRONG = 2,
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_dpc - Defective pixel correction (DPC) parameters set
> > + * with the V4L2_CID_USER_BCM2835_ISP_DPC ctrl.
> > + *
> > + * @enabled: Enable DPC.
> > + * @strength: DPC strength (see enum &bcm2835_isp_dpc_mode).
>
> Isn't DPC_MODE_OFF equal to just setting 'enabled' to false? If so,
> wouldn't the 'strength' field be sufficient?
This is a bit of a quirk of the hardware pipeline. DPC_MODE_OFF still
keeps the block enabled, but running (mostly) as a passthrough. The
enabled field physically switches off the block.
>
> > + */
> > +struct bcm2835_isp_dpc {
> > + __u32 enabled;
> > + __u32 strength;
> > +};
> > +
> > +/*
> > + * ISP statistics structures.
> > + *
> > + * The bcm2835_isp_stats structure is generated at the output of the
> > + * statistics node. Note that this does not directly map onto the statistics
> > + * output of the ISP HW. Instead, the MMAL firmware code maps the HW statistics
> > + * to the bcm2835_isp_stats structure.
> > + */
> > +#define DEFAULT_AWB_REGIONS_X 16
> > +#define DEFAULT_AWB_REGIONS_Y 12
> > +
> > +#define NUM_HISTOGRAMS 2
> > +#define NUM_HISTOGRAM_BINS 128
> > +#define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
> > +#define FLOATING_REGIONS 16
> > +#define AGC_REGIONS 16
> > +#define FOCUS_REGIONS 12
> > +
> > +/**
> > + * struct bcm2835_isp_stats_hist - Histogram statistics
> > + *
> > + * @r_hist: Red channel histogram.
> > + * @g_hist: Combined green channel histogram.
> > + * @b_hist: Blue channel histogram.
> > + */
> > +struct bcm2835_isp_stats_hist {
> > + __u32 r_hist[NUM_HISTOGRAM_BINS];
> > + __u32 g_hist[NUM_HISTOGRAM_BINS];
> > + __u32 b_hist[NUM_HISTOGRAM_BINS];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_stats_region - Region sums.
> > + *
> > + * @counted: The number of 2x2 bayer tiles accumulated.
> > + * @notcounted: The number of 2x2 bayer tiles not accumulated.
> > + * @r_sum: Total sum of counted pixels in the red channel for a region.
> > + * @g_sum: Total sum of counted pixels in the green channel for a region.
> > + * @b_sum: Total sum of counted pixels in the blue channel for a region.
> > + */
> > +struct bcm2835_isp_stats_region {
> > + __u32 counted;
> > + __u32 notcounted;
> > + __u64 r_sum;
> > + __u64 g_sum;
> > + __u64 b_sum;
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_stats_focus - Focus statistics.
> > + *
> > + * @contrast_val: Focus measure - accumulated output of the focus filter.
> > + * In the first dimension, index [0] counts pixels below a
> > + * preset threshold, and index [1] counts pixels above the
> > + * threshold. In the second dimension, index [0] uses the
> > + * first predefined filter, and index [1] uses the second
> > + * predefined filter.
> > + * @contrast_val_num: The number of counted pixels in the above accumulation.
> > + */
> > +struct bcm2835_isp_stats_focus {
> > + __u64 contrast_val[2][2];
> > + __u32 contrast_val_num[2][2];
> > +};
> > +
> > +/**
> > + * struct bcm2835_isp_stats - ISP statistics.
> > + *
> > + * @version: Version of the bcm2835_isp_stats structure.
> > + * @size: Size of the bcm2835_isp_stats structure.
> > + * @hist: Histogram statistics for the entire image.
> > + * @awb_stats: Statistics for the regions defined for AWB calculations.
> > + * @floating_stats: Statistics for arbitrarily placed (floating) regions.
> > + * @agc_stats: Statistics for the regions defined for AGC calculations.
> > + * @focus_stats: Focus filter statistics for the focus regions.
> > + */
> > +struct bcm2835_isp_stats {
> > + __u32 version;
> > + __u32 size;
> > + struct bcm2835_isp_stats_hist hist[NUM_HISTOGRAMS];
> > + struct bcm2835_isp_stats_region awb_stats[AWB_REGIONS];
> > + struct bcm2835_isp_stats_region floating_stats[FLOATING_REGIONS];
> > + struct bcm2835_isp_stats_region agc_stats[AGC_REGIONS];
> > + struct bcm2835_isp_stats_focus focus_stats[FOCUS_REGIONS];
> > +};
> > +
> > +#endif /* __BCM2835_ISP_H_ */
> > diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > index 106f71e709df..072f3c755a68 100644
> > --- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > +++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
> > @@ -5,4 +5,5 @@ config BCM2835_VCHIQ_MMAL
> > help
> > Enables the MMAL API over VCHIQ interface as used for the
> > majority of the multimedia services on VideoCore.
> > - Defaults to Y when the Broadcomd BCM2835 camera host is selected.
> > + Defaults to Y when the Broadcomd BCM2835 camera host or ISP are
> > + selected.
> > diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> > index 44ba91aa6d47..8d904fcce388 100644
> > --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> > +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
> > @@ -100,6 +100,10 @@
> > */
> > #define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
> >
> > +/** ISP image statistics format
> > + */
> > +#define MMAL_ENCODING_BRCM_STATS MMAL_FOURCC('S', 'T', 'A', 'T')
> > +
> > /* }@ */
> >
> > /** \name Pre-defined audio encodings */
> > diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > index 1793103b18fd..b3552af5cf8f 100644
> > --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
> > @@ -221,6 +221,62 @@ enum mmal_parameter_camera_type {
> > MMAL_PARAMETER_SHUTTER_SPEED,
> > /**< Takes a @ref MMAL_PARAMETER_AWB_GAINS_T */
> > MMAL_PARAMETER_CUSTOM_AWB_GAINS,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_SETTINGS_T */
> > + MMAL_PARAMETER_CAMERA_SETTINGS,
> > + /**< Takes a @ref MMAL_PARAMETER_PRIVACY_INDICATOR_T */
> > + MMAL_PARAMETER_PRIVACY_INDICATOR,
> > + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
> > + MMAL_PARAMETER_VIDEO_DENOISE,
> > + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
> > + MMAL_PARAMETER_STILLS_DENOISE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_ANNOTATE_T */
> > + MMAL_PARAMETER_ANNOTATE,
> > + /**< Takes a @ref MMAL_PARAMETER_STEREOSCOPIC_MODE_T */
> > + MMAL_PARAMETER_STEREOSCOPIC_MODE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_INTERFACE_T */
> > + MMAL_PARAMETER_CAMERA_INTERFACE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_CLOCKING_MODE_T */
> > + MMAL_PARAMETER_CAMERA_CLOCKING_MODE,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_CONFIG_T */
> > + MMAL_PARAMETER_CAMERA_RX_CONFIG,
> > + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_TIMING_T */
> > + MMAL_PARAMETER_CAMERA_RX_TIMING,
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_DPF_CONFIG,
> > +
> > + /* 0x50 */
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_JPEG_RESTART_INTERVAL,
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_CAMERA_ISP_BLOCK_OVERRIDE,
> > + /**< Takes a @ref MMAL_PARAMETER_LENS_SHADING_T */
> > + MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
> > + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
> > + MMAL_PARAMETER_BLACK_LEVEL,
> > + /**< Takes a @ref MMAL_PARAMETER_RESIZE_T */
> > + MMAL_PARAMETER_RESIZE_PARAMS,
> > + /**< Takes a @ref MMAL_PARAMETER_CROP_T */
> > + MMAL_PARAMETER_CROP,
> > + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
> > + MMAL_PARAMETER_OUTPUT_SHIFT,
> > + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
> > + MMAL_PARAMETER_CCM_SHIFT,
> > + /**< Takes a @ref MMAL_PARAMETER_CUSTOM_CCM_T */
> > + MMAL_PARAMETER_CUSTOM_CCM,
> > + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
> > + MMAL_PARAMETER_ANALOG_GAIN,
> > + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
> > + MMAL_PARAMETER_DIGITAL_GAIN,
> > + /**< Takes a @ref MMAL_PARAMETER_DENOISE_T */
> > + MMAL_PARAMETER_DENOISE,
> > + /**< Takes a @ref MMAL_PARAMETER_SHARPEN_T */
> > + MMAL_PARAMETER_SHARPEN,
> > + /**< Takes a @ref MMAL_PARAMETER_GEQ_T */
> > + MMAL_PARAMETER_GEQ,
> > + /**< Tales a @ref MMAP_PARAMETER_DPC_T */
> > + MMAL_PARAMETER_DPC,
> > + /**< Tales a @ref MMAP_PARAMETER_GAMMA_T */
> > + MMAL_PARAMETER_GAMMA,
> > };
> >
> > struct mmal_parameter_rational {
> > @@ -779,7 +835,102 @@ struct mmal_parameter_camera_info {
> > struct mmal_parameter_camera_info_camera
> > cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
> > struct mmal_parameter_camera_info_flash
> > - flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
> > + flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
> > +};
> > +
> > +struct mmal_parameter_ccm {
> > + struct mmal_parameter_rational ccm[3][3];
> > + s32 offsets[3];
> > +};
> > +
> > +struct mmal_parameter_custom_ccm {
> > + u32 enabled; /**< Enable the custom CCM. */
> > + struct mmal_parameter_ccm ccm; /**< CCM to be used. */
> > +};
> > +
> > +struct mmal_parameter_lens_shading {
> > + u32 enabled;
> > + u32 grid_cell_size;
> > + u32 grid_width;
> > + u32 grid_stride;
> > + u32 grid_height;
> > + u32 mem_handle_table;
> > + u32 ref_transform;
> > +};
> > +
> > +enum mmal_parameter_ls_gain_format_type {
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U0P8_1 = 0,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_0 = 1,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_1 = 2,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_0 = 3,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_1 = 4,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_0 = 5,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_1 = 6,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U4P10 = 7,
> > + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_DUMMY = 0x7FFFFFFF
> > +};
> > +
> > +struct mmal_parameter_lens_shading_v2 {
> > + u32 enabled;
> > + u32 grid_cell_size;
> > + u32 grid_width;
> > + u32 grid_stride;
> > + u32 grid_height;
> > + u32 mem_handle_table;
> > + u32 ref_transform;
> > + u32 corner_sampled;
> > + enum mmal_parameter_ls_gain_format_type gain_format;
> > +};
> > +
> > +struct mmal_parameter_black_level {
> > + u32 enabled;
> > + u16 black_level_r;
> > + u16 black_level_g;
> > + u16 black_level_b;
> > + u8 pad_[2]; /* Unused */
> > +};
> > +
> > +struct mmal_parameter_geq {
> > + u32 enabled;
> > + u32 offset;
> > + struct mmal_parameter_rational slope;
> > +};
> > +
> > +#define MMAL_NUM_GAMMA_PTS 33
> > +struct mmal_parameter_gamma {
> > + u32 enabled;
> > + u16 x[MMAL_NUM_GAMMA_PTS];
> > + u16 y[MMAL_NUM_GAMMA_PTS];
> > +};
> > +
> > +struct mmal_parameter_denoise {
> > + u32 enabled;
> > + u32 constant;
> > + struct mmal_parameter_rational slope;
> > + struct mmal_parameter_rational strength;
> > +};
> > +
> > +struct mmal_parameter_sharpen {
> > + u32 enabled;
> > + struct mmal_parameter_rational threshold;
> > + struct mmal_parameter_rational strength;
> > + struct mmal_parameter_rational limit;
> > +};
> > +
> > +enum mmal_dpc_mode {
> > + MMAL_DPC_MODE_OFF = 0,
> > + MMAL_DPC_MODE_NORMAL = 1,
> > + MMAL_DPC_MODE_STRONG = 2,
> > + MMAL_DPC_MODE_MAX = 0x7FFFFFFF,
> > +};
> > +
> > +struct mmal_parameter_dpc {
> > + u32 enabled;
> > + u32 strength;
> > +};
> > +
> > +struct mmal_parameter_crop {
> > + struct vchiq_mmal_rect rect;
> > };
> >
> > #endif
> >
>
> Regards,
>
> Hans
Regards,
Naush
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-19 12:47 ` Naushir Patuck
@ 2020-05-19 13:03 ` Jacopo Mondi
2020-06-24 11:28 ` Hans Verkuil
1 sibling, 0 replies; 111+ messages in thread
From: Jacopo Mondi @ 2020-05-19 13:03 UTC (permalink / raw)
To: Naushir Patuck
Cc: Hans Verkuil, Laurent Pinchart, linux-media, Kieran Bingham,
Niklas Söderlund, Dave Stevenson
Hi,
On Tue, May 19, 2020 at 01:47:16PM +0100, Naushir Patuck wrote:
[snip]
> > > +++ b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
> > > @@ -0,0 +1,333 @@
> > > +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
> > > +/*
> > > + * bcm2835-isp.h
> > > + *
> > > + * BCM2835 ISP driver - user space header file.
> > > + *
> > > + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
> > > + *
> > > + * Author: Naushir Patuck (naush@raspberrypi.com)
> > > + *
> > > + */
> > > +
> > > +#ifndef __BCM2835_ISP_H_
> > > +#define __BCM2835_ISP_H_
> > > +
> > > +#include <linux/v4l2-controls.h>
> > > +
> > > +/* TODO: move the control IDs definitions to v4l2-controls.h */
> > > +#define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10c0)
> >
> > As the TODO says: move this to v4l2-controls.h. Currently the 0x10c0 offset
> > clashes with V4L2_CID_USER_ATMEL_ISC_BASE, so that certainly should be fixed.
> >
>
> Unfortunately, there seems to be a mixup here. Laurent, we have
> accidentally mailed a WIP revision of this patch. The final version
> does have V4L2_CID_USER_BCM2835_ISP_BASE with a unique id in
> v4l2-controls.h. I will talk with Laurent separately to get the
> correct revison included in the next patch-set.
I have created this file in this directory with the intention not to
pollute the kernel headers with definitions for a staging driver,
like in example, the IPU3 definitions in
drivers/staging/media/ipu3/include/intel-ipu3.h
Which, by the way, has another conflicting definition for its control
base
define V4L2_CID_INTEL_IPU3_BASE (V4L2_CID_USER_BASE + 0x10c0)
>
> > > +
> > > +/* TODO: move the formats definitions to videodev2.h */
> > > +/* 12 Y/CbCr 4:2:0 128 pixel wide column */
> > > +#define V4L2_PIX_FMT_NV12_COL128 v4l2_fourcc('N', 'C', '1', '2')
> > > +/* Y/CbCr 4:2:0 10bpc, 3x10 packed as 4 bytes in a 128 bytes / 96 pixel wide column */
> > > +#define V4L2_PIX_FMT_NV12_10_COL128 v4l2_fourcc('N', 'C', '3', '0')
> > > +/* Sensor Ancillary metadata */
> > > +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S')
> > > +/* BCM2835 ISP image statistics output */
> > > +#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A')
> > > +
>
> Similarly, these have also been moved to the right header files.
Same. I'm not sure now what the right policy for a staging driver
should be then.
>
> > > +#define V4L2_CID_USER_BCM2835_ISP_CC_MATRIX \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0001)
> > > +#define V4L2_CID_USER_BCM2835_ISP_LENS_SHADING \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0002)
> > > +#define V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0003)
> > > +#define V4L2_CID_USER_BCM2835_ISP_GEQ \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0004)
> > > +#define V4L2_CID_USER_BCM2835_ISP_GAMMA \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0005)
> > > +#define V4L2_CID_USER_BCM2835_ISP_DENOISE \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0006)
> > > +#define V4L2_CID_USER_BCM2835_ISP_SHARPEN \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
> > > +#define V4L2_CID_USER_BCM2835_ISP_DPC \
> > > + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
> >
> > There is no documentation for these controls. Specifically, it doesn't
> > tell you which struct should be used.
>
> As above, the documentaiton is available in the newer patch.
>
related: the documentation for this driver is in
drivers/staging/vc04_services/Documentation/
should this be moved to Documentation already ?
(as Nuash said, there's no documentation for controls in this version)
Thanks
j
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP
2020-05-19 12:47 ` Naushir Patuck
2020-05-19 13:03 ` Jacopo Mondi
@ 2020-06-24 11:28 ` Hans Verkuil
1 sibling, 0 replies; 111+ messages in thread
From: Hans Verkuil @ 2020-06-24 11:28 UTC (permalink / raw)
To: Naushir Patuck
Cc: Laurent Pinchart, linux-media, Kieran Bingham, Jacopo Mondi,
Niklas Söderlund, Dave Stevenson
On 19/05/2020 14:47, Naushir Patuck wrote:
> Hi Hans,
>
> Thank you for the review and comments. Just a follow up and
> overlapping Dave's comments:
>
> On Mon, 18 May 2020 at 13:02, Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 04/05/2020 11:26, Laurent Pinchart wrote:
>>> From: Naushir Patuck <naush@raspberrypi.com>
>>>
>>> Port the V4L2 compatible driver for the ISP unit found on Broadcom BCM2835
>>> chips.
>>>
>>> The driver interfaces though the VideoCore unit using the VCHIQ MMAL
>>> interface.
>>>
>>> ISP driver upported from from RaspberryPi BSP at revision:
>>> 6c3505be6c3e ("staging: vc04_services: isp: Make all references to bcm2835_isp_fmt const")
>>>
>>> Signed-off-by: Dave Stevenson <dave.stevenson@raspberrypi.com>
>>> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
>>> [Adapt to staging by moving all modifications that in the BSP are scattered
>>> in core components inside this directory]
>>> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
>>> ---
>>> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 41 +
>>> drivers/staging/vc04_services/Kconfig | 2 +
>>> drivers/staging/vc04_services/Makefile | 1 +
>>> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
>>> .../vc04_services/bcm2835-isp/Makefile | 10 +
>>> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1632 +++++++++++++++++
>>> .../bcm2835-isp/bcm2835_isp_ctrls.h | 67 +
>>> .../bcm2835-isp/bcm2835_isp_fmts.h | 301 +++
>>> .../include/uapi/linux/bcm2835-isp.h | 333 ++++
>>> .../staging/vc04_services/vchiq-mmal/Kconfig | 3 +-
>>> .../vc04_services/vchiq-mmal/mmal-encodings.h | 4 +
>>> .../vchiq-mmal/mmal-parameters.h | 153 +-
>>> 12 files changed, 2559 insertions(+), 2 deletions(-)
>>> create mode 100644 drivers/staging/vc04_services/Documentation/userspace-api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
>>> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
>>> create mode 100644 drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
>>>
>>
>> <snip>
>>
>>> diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>>> new file mode 100644
>>> index 000000000000..a32faab4b8dc
>>> --- /dev/null
>>> +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-isp.c
>>> @@ -0,0 +1,1632 @@
>>
<snip>
>>
>>> + break;
>>> + }
>>> +
>>> + /* We use the selection API instead of the old crop API. */
>>> + v4l2_disable_ioctl(vfd, VIDIOC_CROPCAP);
>>> + v4l2_disable_ioctl(vfd, VIDIOC_G_CROP);
>>> + v4l2_disable_ioctl(vfd, VIDIOC_S_CROP);
>>
>> No need for this: the core handles this and will disable these ioctls
>> automatically.
>
> Without these explicitally disabled, we fail v4l2-compliance on the
> video capture nodes with the following error:
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> fail: v4l2-test-formats.cpp(1510): doioctl(node, VIDIOC_G_SELECTION, &sel)
I believe that's because G_SELECTION doesn't support the V4L2_SEL_TGT_CROP_DEFAULT
target. Once that is supported, this test should pass.
Regards,
Hans
> fail: v4l2-test-formats.cpp(1550): testLegacyCrop(node)
> test Cropping: FAIL
> test Composing: OK (Not Supported)
> test Scaling: OK (Not Supported)
>
>>
>>> +
>>> + ret = bcm2835_isp_get_supported_fmts(node);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + /* Initialise the the video node. */
>>> + vfd->vfl_type = VFL_TYPE_VIDEO;
>>> + vfd->fops = &bcm2835_isp_fops,
>>> + vfd->ioctl_ops = &bcm2835_isp_node_ioctl_ops,
>>> + vfd->minor = -1,
>>> + vfd->release = video_device_release_empty,
>>> + vfd->queue = &node->queue;
>>> + vfd->lock = &node->lock;
>>> + vfd->v4l2_dev = &dev->v4l2_dev;
>>> + vfd->vfl_dir = node->vfl_dir;
>>> +
>>> + node->q_data.fmt = get_default_format(node);
>>> + node->q_data.width = DEFAULT_DIM;
>>> + node->q_data.height = DEFAULT_DIM;
>>> + node->q_data.bytesperline =
>>> + get_bytesperline(DEFAULT_DIM, node->q_data.fmt);
>>> + node->q_data.sizeimage = node_is_stats(node) ?
>>> + get_port_data(node)->recommended_buffer.size :
>>> + get_sizeimage(node->q_data.bytesperline,
>>> + node->q_data.width,
>>> + node->q_data.height,
>>> + node->q_data.fmt);
>>> +
>>> + queue->io_modes = VB2_MMAP | VB2_DMABUF;
>>> + queue->drv_priv = node;
>>> + queue->ops = &bcm2835_isp_node_queue_ops;
>>> + queue->mem_ops = &vb2_dma_contig_memops;
>>> + queue->buf_struct_size = sizeof(struct bcm2835_isp_buffer);
>>> + queue->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
>>> + queue->dev = dev->dev;
>>> + queue->lock = &node->queue_lock;
>>> +
>>> + ret = vb2_queue_init(queue);
>>> + if (ret < 0) {
>>> + v4l2_info(&dev->v4l2_dev, "vb2_queue_init failed\n");
>>> + return ret;
>>> + }
>>> + node->queue_init = true;
>>> +
>>> + /* Define the device names */
>>> + snprintf(vfd->name, sizeof(node->vfd.name), "%s-%s%d", BCM2835_ISP_NAME,
>>> + node->name, node->id);
>>> +
>>> + ret = video_register_device(vfd, VFL_TYPE_VIDEO, video_nr + index);
>>> + if (ret) {
>>> + v4l2_err(&dev->v4l2_dev,
>>> + "Failed to register video %s[%d] device node\n",
>>> + node->name, node->id);
>>> + return ret;
>>> + }
>>
>> Move registering the video device to the end of this function.
>> Otherwise the output video device would be created (and available for
>> userspace) before the controls are added.
>>
>
> Ack
>
>>> +
>>> + node->registered = true;
>>> + video_set_drvdata(vfd, node);
>>> +
>>> + /* Set some controls and defaults, but only on the VIDEO_OUTPUT node. */
>>> + if (node_is_output(node)) {
>>> + unsigned int i;
>>> +
>>> + /* Use this ctrl template to assign all out ISP custom ctrls. */
>>> + struct v4l2_ctrl_config ctrl_template = {
>>> + .ops = &bcm2835_isp_ctrl_ops,
>>> + .type = V4L2_CTRL_TYPE_U8,
>>> + .def = 0,
>>> + .min = 0x00,
>>> + .max = 0xff,
>>> + .step = 1,
>>> + };
>>> +
>>> + v4l2_ctrl_handler_init(&dev->ctrl_handler, 4);
>>> +
>>> + dev->r_gain = 1000;
>>> + dev->b_gain = 1000;
>>> +
>>> + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
>>> + V4L2_CID_RED_BALANCE, 1, 0xffff, 1,
>>> + dev->r_gain);
>>> +
>>> + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
>>> + V4L2_CID_BLUE_BALANCE, 1, 0xffff, 1,
>>> + dev->b_gain);
>>> +
>>> + v4l2_ctrl_new_std(&dev->ctrl_handler, &bcm2835_isp_ctrl_ops,
>>> + V4L2_CID_DIGITAL_GAIN, 1, 0xffff, 1, 1000);
>>> +
>>> + for (i = 0; i < ARRAY_SIZE(custom_ctrls); i++) {
>>> + ctrl_template.name = custom_ctrls[i].name;
>>> + ctrl_template.id = custom_ctrls[i].id;
>>> + ctrl_template.dims[0] = custom_ctrls[i].size;
>>> + ctrl_template.flags = custom_ctrls[i].flags;
>>> + v4l2_ctrl_new_custom(&dev->ctrl_handler,
>>> + &ctrl_template, NULL);
>>> + }
>>> +
>>> + node->vfd.ctrl_handler = &dev->ctrl_handler;
>>
>> Missing error check.
>
> Ack
>
>>
>>> + }
>>> +
>>> + v4l2_info(&dev->v4l2_dev,
>>> + "Device node %s[%d] registered as /dev/video%d\n",
>>> + node->name, node->id, vfd->num);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +/* Unregister one of the /dev/video<N> nodes associated with the ISP. */
>>> +static void unregister_node(struct bcm2835_isp_node *node)
>>> +{
>>> + struct bcm2835_isp_dev *dev = node_get_dev(node);
>>> +
>>> + v4l2_info(&dev->v4l2_dev,
>>> + "Unregistering node %s[%d] device node /dev/video%d\n",
>>> + node->name, node->id, node->vfd.num);
>>> +
>>> + if (node->queue_init)
>>> + vb2_queue_release(&node->queue);
>>> +
>>> + if (node->registered) {
>>> + video_unregister_device(&node->vfd);
>>> + if (node_is_output(node))
>>> + v4l2_ctrl_handler_free(&dev->ctrl_handler);
>>> + }
>>> +
>>> + /*
>>> + * node->supported_fmts.list is free'd automatically
>>> + * as a managed resource.
>>> + */
>>> + node->supported_fmts.list = NULL;
>>> + node->supported_fmts.num_entries = 0;
>>> + node->vfd.ctrl_handler = NULL;
>>> + node->registered = false;
>>> + node->queue_init = false;
>>> +}
>>> +
>>> +static void media_controller_unregister(struct bcm2835_isp_dev *dev)
>>> +{
>>> + unsigned int i;
>>> +
>>> + v4l2_info(&dev->v4l2_dev, "Unregister from media controller\n");
>>> +
>>> + if (dev->media_device_registered) {
>>> + media_device_unregister(&dev->mdev);
>>> + media_device_cleanup(&dev->mdev);
>>> + dev->media_device_registered = false;
>>> + }
>>> +
>>> + kfree(dev->entity.name);
>>> + dev->entity.name = NULL;
>>> +
>>> + if (dev->media_entity_registered) {
>>> + media_device_unregister_entity(&dev->entity);
>>> + dev->media_entity_registered = false;
>>> + }
>>> +
>>> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
>>> + struct bcm2835_isp_node *node = &dev->node[i];
>>> +
>>> + if (node->media_node_registered) {
>>> + media_remove_intf_links(node->intf_link->intf);
>>> + media_entity_remove_links(&dev->node[i].vfd.entity);
>>> + media_devnode_remove(node->intf_devnode);
>>> + media_device_unregister_entity(&node->vfd.entity);
>>> + kfree(node->vfd.entity.name);
>>> + }
>>> + node->media_node_registered = false;
>>> + }
>>> +
>>> + dev->v4l2_dev.mdev = NULL;
>>> +}
>>> +
>>> +static int media_controller_register_node(struct bcm2835_isp_dev *dev, int num)
>>> +{
>>> + struct bcm2835_isp_node *node = &dev->node[num];
>>> + struct media_entity *entity = &node->vfd.entity;
>>> + int output = node_is_output(node);
>>> + char *name;
>>> + int ret;
>>> +
>>> + v4l2_info(&dev->v4l2_dev,
>>> + "Register %s node %d with media controller\n",
>>> + output ? "output" : "capture", num);
>>> + entity->obj_type = MEDIA_ENTITY_TYPE_VIDEO_DEVICE;
>>> + entity->function = MEDIA_ENT_F_IO_V4L;
>>> + entity->info.dev.major = VIDEO_MAJOR;
>>> + entity->info.dev.minor = node->vfd.minor;
>>> + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
>>> + if (!name) {
>>> + ret = -ENOMEM;
>>> + goto error_no_mem;
>>> + }
>>> + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "%s0-%s%d",
>>> + BCM2835_ISP_NAME, output ? "output" : "capture", num);
>>> + entity->name = name;
>>> + node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK;
>>> + ret = media_entity_pads_init(entity, 1, &node->pad);
>>> + if (ret)
>>> + goto error_pads_init;
>>> + ret = media_device_register_entity(&dev->mdev, entity);
>>> + if (ret)
>>> + goto error_register_entity;
>>> +
>>> + node->intf_devnode = media_devnode_create(&dev->mdev,
>>> + MEDIA_INTF_T_V4L_VIDEO, 0,
>>> + VIDEO_MAJOR, node->vfd.minor);
>>> + if (!node->intf_devnode) {
>>> + ret = -ENOMEM;
>>> + goto error_devnode_create;
>>> + }
>>> +
>>> + node->intf_link = media_create_intf_link(entity,
>>> + &node->intf_devnode->intf,
>>> + MEDIA_LNK_FL_IMMUTABLE |
>>> + MEDIA_LNK_FL_ENABLED);
>>> + if (!node->intf_link) {
>>> + ret = -ENOMEM;
>>> + goto error_create_intf_link;
>>> + }
>>> +
>>> + if (output)
>>> + ret = media_create_pad_link(entity, 0, &dev->entity, num,
>>> + MEDIA_LNK_FL_IMMUTABLE |
>>> + MEDIA_LNK_FL_ENABLED);
>>> + else
>>> + ret = media_create_pad_link(&dev->entity, num, entity, 0,
>>> + MEDIA_LNK_FL_IMMUTABLE |
>>> + MEDIA_LNK_FL_ENABLED);
>>> + if (ret)
>>> + goto error_create_pad_link;
>>> +
>>> + dev->node[num].media_node_registered = true;
>>> + return 0;
>>> +
>>> +error_create_pad_link:
>>> + media_remove_intf_links(&node->intf_devnode->intf);
>>> +error_create_intf_link:
>>> + media_devnode_remove(node->intf_devnode);
>>> +error_devnode_create:
>>> + media_device_unregister_entity(&node->vfd.entity);
>>> +error_register_entity:
>>> +error_pads_init:
>>> + kfree(entity->name);
>>> + entity->name = NULL;
>>> +error_no_mem:
>>> + if (ret)
>>> + v4l2_info(&dev->v4l2_dev, "Error registering node\n");
>>> +
>>> + return ret;
>>> +}
>>> +
>>> +static int media_controller_register(struct bcm2835_isp_dev *dev)
>>> +{
>>> + char *name;
>>> + unsigned int i;
>>> + int ret;
>>> +
>>> + v4l2_dbg(2, debug, &dev->v4l2_dev, "Registering with media controller\n");
>>> + dev->mdev.dev = dev->dev;
>>> + strscpy(dev->mdev.model, "bcm2835-isp",
>>> + sizeof(dev->mdev.model));
>>> + strscpy(dev->mdev.bus_info, "platform:bcm2835-isp",
>>> + sizeof(dev->mdev.bus_info));
>>> + media_device_init(&dev->mdev);
>>> + dev->v4l2_dev.mdev = &dev->mdev;
>>> +
>>> + v4l2_dbg(2, debug, &dev->v4l2_dev, "Register entity for nodes\n");
>>> +
>>> + name = kmalloc(BCM2835_ISP_ENTITY_NAME_LEN, GFP_KERNEL);
>>> + if (!name) {
>>> + ret = -ENOMEM;
>>> + goto done;
>>> + }
>>> + snprintf(name, BCM2835_ISP_ENTITY_NAME_LEN, "bcm2835_isp0");
>>> + dev->entity.name = name;
>>> + dev->entity.obj_type = MEDIA_ENTITY_TYPE_BASE;
>>> + dev->entity.function = MEDIA_ENT_F_PROC_VIDEO_SCALER;
>>> +
>>> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
>>> + dev->pad[i].flags = node_is_output(&dev->node[i]) ?
>>> + MEDIA_PAD_FL_SINK : MEDIA_PAD_FL_SOURCE;
>>> + }
>>> +
>>> + ret = media_entity_pads_init(&dev->entity, BCM2835_ISP_NUM_NODES,
>>> + dev->pad);
>>> + if (ret)
>>> + goto done;
>>> +
>>> + ret = media_device_register_entity(&dev->mdev, &dev->entity);
>>> + if (ret)
>>> + goto done;
>>> +
>>> + dev->media_entity_registered = true;
>>> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
>>> + ret = media_controller_register_node(dev, i);
>>> + if (ret)
>>> + goto done;
>>> + }
>>> +
>>> + ret = media_device_register(&dev->mdev);
>>> + if (!ret)
>>> + dev->media_device_registered = true;
>>> +done:
>>> + return ret;
>>> +}
>>> +
>>> +static int bcm2835_isp_remove(struct platform_device *pdev)
>>> +{
>>> + struct bcm2835_isp_dev *dev = platform_get_drvdata(pdev);
>>> + unsigned int i;
>>> +
>>> + media_controller_unregister(dev);
>>> +
>>> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++)
>>> + unregister_node(&dev->node[i]);
>>> +
>>> + v4l2_device_unregister(&dev->v4l2_dev);
>>> +
>>> + if (dev->component)
>>> + vchiq_mmal_component_finalise(dev->mmal_instance,
>>> + dev->component);
>>> +
>>> + vchiq_mmal_finalise(dev->mmal_instance);
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int bcm2835_isp_probe(struct platform_device *pdev)
>>> +{
>>> + struct bcm2835_isp_dev *dev;
>>> + unsigned int i;
>>> + int ret;
>>> +
>>> + dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
>>> + if (!dev)
>>> + return -ENOMEM;
>>> +
>>> + dev->dev = &pdev->dev;
>>> +
>>> + ret = v4l2_device_register(&pdev->dev, &dev->v4l2_dev);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + ret = vchiq_mmal_init(&dev->mmal_instance);
>>> + if (ret) {
>>> + v4l2_device_unregister(&dev->v4l2_dev);
>>> + return ret;
>>> + }
>>> +
>>> + ret = vchiq_mmal_component_init(dev->mmal_instance, "ril.isp",
>>> + &dev->component);
>>> + if (ret) {
>>> + v4l2_err(&dev->v4l2_dev,
>>> + "%s: failed to create ril.isp component\n", __func__);
>>> + goto error;
>>> + }
>>> +
>>> + if ((dev->component->inputs != BCM2835_ISP_NUM_OUTPUTS) ||
>>> + (dev->component->outputs != BCM2835_ISP_NUM_CAPTURES +
>>> + BCM2835_ISP_NUM_METADATA)) {
>>> + v4l2_err(&dev->v4l2_dev,
>>> + "%s: ril.isp returned %d i/p (%d expected), %d o/p (%d expected) ports\n",
>>> + __func__, dev->component->inputs,
>>> + BCM2835_ISP_NUM_OUTPUTS,
>>> + dev->component->outputs,
>>> + BCM2835_ISP_NUM_CAPTURES + BCM2835_ISP_NUM_METADATA);
>>> + goto error;
>>> + }
>>> +
>>> + atomic_set(&dev->num_streaming, 0);
>>> +
>>> + for (i = 0; i < BCM2835_ISP_NUM_NODES; i++) {
>>> + struct bcm2835_isp_node *node = &dev->node[i];
>>> +
>>> + ret = register_node(dev, node, i);
>>> + if (ret)
>>> + goto error;
>>> + }
>>> +
>>> + ret = media_controller_register(dev);
>>> + if (ret)
>>> + goto error;
>>> +
>>> + platform_set_drvdata(pdev, dev);
>>> + v4l2_info(&dev->v4l2_dev, "Loaded V4L2 %s\n", BCM2835_ISP_NAME);
>>> + return 0;
>>> +
>>> +error:
>>> + bcm2835_isp_remove(pdev);
>>> +
>>> + return ret;
>>> +}
>>> +
>>> +static struct platform_driver bcm2835_isp_pdrv = {
>>> + .probe = bcm2835_isp_probe,
>>> + .remove = bcm2835_isp_remove,
>>> + .driver = {
>>> + .name = BCM2835_ISP_NAME,
>>> + },
>>> +};
>>> +
>>> +module_platform_driver(bcm2835_isp_pdrv);
>>> +
>>> +MODULE_DESCRIPTION("BCM2835 ISP driver");
>>> +MODULE_AUTHOR("Naushir Patuck <naush@raspberrypi.com>");
>>> +MODULE_LICENSE("GPL");
>>> +MODULE_VERSION("1.0");
>>> +MODULE_ALIAS("platform:bcm2835-isp");
>>> diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
>>> new file mode 100644
>>> index 000000000000..cfbb1063aad1
>>> --- /dev/null
>>> +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_ctrls.h
>>> @@ -0,0 +1,67 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Broadcom BCM2835 ISP driver
>>> + *
>>> + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
>>> + *
>>> + * Author: Naushir Patuck (naush@raspberrypi.com)
>>> + *
>>> + */
>>> +
>>> +#ifndef BCM2835_ISP_CTRLS
>>> +#define BCM2835_ISP_CTRLS
>>> +
>>> +#include <linux/bcm2835-isp.h>
>>> +
>>> +struct bcm2835_isp_custom_ctrl {
>>> + const char *name;
>>> + u32 id;
>>> + u32 size;
>>> + u32 flags;
>>> +};
>>> +
>>> +static const struct bcm2835_isp_custom_ctrl custom_ctrls[] = {
>>> + {
>>> + .name = "Colour Correction Matrix",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_CC_MATRIX,
>>> + .size = sizeof(struct bcm2835_isp_custom_ccm),
>>> + .flags = 0
>>> + }, {
>>> + .name = "Lens Shading",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_LENS_SHADING,
>>> + .size = sizeof(struct bcm2835_isp_lens_shading),
>>> + .flags = V4L2_CTRL_FLAG_EXECUTE_ON_WRITE
>>> + }, {
>>> + .name = "Black Level",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL,
>>> + .size = sizeof(struct bcm2835_isp_black_level),
>>> + .flags = 0
>>> + }, {
>>> + .name = "Green Equalisation",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_GEQ,
>>> + .size = sizeof(struct bcm2835_isp_geq),
>>> + .flags = 0
>>> + }, {
>>> + .name = "Gamma",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_GAMMA,
>>> + .size = sizeof(struct bcm2835_isp_gamma),
>>> + .flags = 0
>>> + }, {
>>> + .name = "Sharpen",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_SHARPEN,
>>> + .size = sizeof(struct bcm2835_isp_sharpen),
>>> + .flags = 0
>>> + }, {
>>> + .name = "Denoise",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_DENOISE,
>>> + .size = sizeof(struct bcm2835_isp_denoise),
>>> + .flags = 0
>>> + }, {
>>> + .name = "Defective Pixel Correction",
>>> + .id = V4L2_CID_USER_BCM2835_ISP_DPC,
>>> + .size = sizeof(struct bcm2835_isp_dpc),
>>> + .flags = 0
>>> + }
>>> +};
>>> +
>>> +#endif
>>> diff --git a/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
>>> new file mode 100644
>>> index 000000000000..af3bde152bb2
>>> --- /dev/null
>>> +++ b/drivers/staging/vc04_services/bcm2835-isp/bcm2835_isp_fmts.h
>>> @@ -0,0 +1,301 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +/*
>>> + * Broadcom BCM2835 ISP driver
>>> + *
>>> + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
>>> + *
>>> + * Author: Naushir Patuck (naush@raspberrypi.com)
>>> + *
>>> + */
>>> +
>>> +#ifndef BCM2835_ISP_FMTS
>>> +#define BCM2835_ISP_FMTS
>>> +
>>> +#include <linux/videodev2.h>
>>> +#include "vchiq-mmal/mmal-encodings.h"
>>> +
>>> +struct bcm2835_isp_fmt {
>>> + u32 fourcc;
>>> + int depth;
>>> + int bytesperline_align;
>>> + u32 flags;
>>> + u32 mmal_fmt;
>>> + int size_multiplier_x2;
>>> + enum v4l2_colorspace colorspace;
>>> + unsigned int step_size;
>>> +};
>>> +
>>> +struct bcm2835_isp_fmt_list {
>>> + struct bcm2835_isp_fmt const **list;
>>> + unsigned int num_entries;
>>> +};
>>> +
>>> +static const struct bcm2835_isp_fmt supported_formats[] = {
>>> + {
>>> + /* YUV formats */
>>> + .fourcc = V4L2_PIX_FMT_YUV420,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_I420,
>>> + .size_multiplier_x2 = 3,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_YVU420,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_YV12,
>>> + .size_multiplier_x2 = 3,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_NV12,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_NV12,
>>> + .size_multiplier_x2 = 3,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_NV21,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_NV21,
>>> + .size_multiplier_x2 = 3,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_YUYV,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_YUYV,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_UYVY,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_UYVY,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_YVYU,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_YVYU,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_VYUY,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_VYUY,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SMPTE170M,
>>> + .step_size = 2,
>>> + }, {
>>> + /* RGB formats */
>>> + .fourcc = V4L2_PIX_FMT_RGB24,
>>> + .depth = 24,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_RGB24,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SRGB,
>>> + .step_size = 1,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_RGB565,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_RGB16,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SRGB,
>>> + .step_size = 1,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_BGR24,
>>> + .depth = 24,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BGR24,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SRGB,
>>> + .step_size = 1,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_ABGR32,
>>> + .depth = 32,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BGRA,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_SRGB,
>>> + .step_size = 1,
>>> + }, {
>>> + /* Bayer formats */
>>> + /* 8 bit */
>>> + .fourcc = V4L2_PIX_FMT_SRGGB8,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB8,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SBGGR8,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR8,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGRBG8,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG8,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGBRG8,
>>> + .depth = 8,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG8,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + /* 10 bit */
>>> + .fourcc = V4L2_PIX_FMT_SRGGB10P,
>>> + .depth = 10,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB10P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SBGGR10P,
>>> + .depth = 10,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR10P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGRBG10P,
>>> + .depth = 10,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG10P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGBRG10P,
>>> + .depth = 10,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG10P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + /* 12 bit */
>>> + .fourcc = V4L2_PIX_FMT_SRGGB12P,
>>> + .depth = 12,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB12P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SBGGR12P,
>>> + .depth = 12,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR12P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGRBG12P,
>>> + .depth = 12,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG12P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGBRG12P,
>>> + .depth = 12,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG12P,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + /* 16 bit */
>>> + .fourcc = V4L2_PIX_FMT_SRGGB16,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SRGGB16,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SBGGR16,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SBGGR16,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGRBG16,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGRBG16,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + .fourcc = V4L2_PIX_FMT_SGBRG16,
>>> + .depth = 16,
>>> + .bytesperline_align = 32,
>>> + .flags = 0,
>>> + .mmal_fmt = MMAL_ENCODING_BAYER_SGBRG16,
>>> + .size_multiplier_x2 = 2,
>>> + .colorspace = V4L2_COLORSPACE_RAW,
>>> + .step_size = 2,
>>> + }, {
>>> + /* ISP statistics format */
>>> + .fourcc = V4L2_META_FMT_BCM2835_ISP_STATS,
>>> + .mmal_fmt = MMAL_ENCODING_BRCM_STATS,
>>> + /* The rest are not valid fields for stats. */
>>> + }
>>> +};
>>> +
>>> +#endif
>>> diff --git a/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
>>> new file mode 100644
>>> index 000000000000..edc452fa8318
>>> --- /dev/null
>>> +++ b/drivers/staging/vc04_services/include/uapi/linux/bcm2835-isp.h
>>> @@ -0,0 +1,333 @@
>>> +/* SPDX-License-Identifier: ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) */
>>> +/*
>>> + * bcm2835-isp.h
>>> + *
>>> + * BCM2835 ISP driver - user space header file.
>>> + *
>>> + * Copyright © 2019-2020 Raspberry Pi (Trading) Ltd.
>>> + *
>>> + * Author: Naushir Patuck (naush@raspberrypi.com)
>>> + *
>>> + */
>>> +
>>> +#ifndef __BCM2835_ISP_H_
>>> +#define __BCM2835_ISP_H_
>>> +
>>> +#include <linux/v4l2-controls.h>
>>> +
>>> +/* TODO: move the control IDs definitions to v4l2-controls.h */
>>> +#define V4L2_CID_USER_BCM2835_ISP_BASE (V4L2_CID_USER_BASE + 0x10c0)
>>
>> As the TODO says: move this to v4l2-controls.h. Currently the 0x10c0 offset
>> clashes with V4L2_CID_USER_ATMEL_ISC_BASE, so that certainly should be fixed.
>>
>
> Unfortunately, there seems to be a mixup here. Laurent, we have
> accidentally mailed a WIP revision of this patch. The final version
> does have V4L2_CID_USER_BCM2835_ISP_BASE with a unique id in
> v4l2-controls.h. I will talk with Laurent separately to get the
> correct revison included in the next patch-set.
>
>>> +
>>> +/* TODO: move the formats definitions to videodev2.h */
>>> +/* 12 Y/CbCr 4:2:0 128 pixel wide column */
>>> +#define V4L2_PIX_FMT_NV12_COL128 v4l2_fourcc('N', 'C', '1', '2')
>>> +/* Y/CbCr 4:2:0 10bpc, 3x10 packed as 4 bytes in a 128 bytes / 96 pixel wide column */
>>> +#define V4L2_PIX_FMT_NV12_10_COL128 v4l2_fourcc('N', 'C', '3', '0')
>>> +/* Sensor Ancillary metadata */
>>> +#define V4L2_META_FMT_SENSOR_DATA v4l2_fourcc('S', 'E', 'N', 'S')
>>> +/* BCM2835 ISP image statistics output */
>>> +#define V4L2_META_FMT_BCM2835_ISP_STATS v4l2_fourcc('B', 'S', 'T', 'A')
>>> +
>
> Similarly, these have also been moved to the right header files.
>
>>> +#define V4L2_CID_USER_BCM2835_ISP_CC_MATRIX \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0001)
>>> +#define V4L2_CID_USER_BCM2835_ISP_LENS_SHADING \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0002)
>>> +#define V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0003)
>>> +#define V4L2_CID_USER_BCM2835_ISP_GEQ \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0004)
>>> +#define V4L2_CID_USER_BCM2835_ISP_GAMMA \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0005)
>>> +#define V4L2_CID_USER_BCM2835_ISP_DENOISE \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0006)
>>> +#define V4L2_CID_USER_BCM2835_ISP_SHARPEN \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0007)
>>> +#define V4L2_CID_USER_BCM2835_ISP_DPC \
>>> + (V4L2_CID_USER_BCM2835_ISP_BASE + 0x0008)
>>
>> There is no documentation for these controls. Specifically, it doesn't
>> tell you which struct should be used.
>
> As above, the documentaiton is available in the newer patch.
>
>>
>>> +
>>> +/*
>>> + * All structs below are directly mapped onto the equivalent structs in
>>> + * drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
>>> + * for convenience.
>>> + */
>>> +
>>> +/**
>>> + * struct bcm2835_isp_rational - Rational value type.
>>> + *
>>> + * @num: Numerator.
>>> + * @den: Denominator.
>>> + */
>>> +struct bcm2835_isp_rational {
>>> + __s32 num;
>>> + __s32 den;
>>
>> Wouldn't it make more sense if den is a __u32?
>
> Ack
>
>>
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_ccm - Colour correction matrix.
>>> + *
>>> + * @ccm: 3x3 correction matrix coefficients.
>>> + * @offsets: 1x3 correction offsets.
>>> + */
>>> +struct bcm2835_isp_ccm {
>>> + struct bcm2835_isp_rational ccm[3][3];
>>> + __s32 offsets[3];
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_custom_ccm - Custom CCM applied with the
>>> + * V4L2_CID_USER_BCM2835_ISP_CC_MATRIX ctrl.
>>> + *
>>> + * @enabled: Enable custom CCM.
>>> + * @ccm: Custom CCM coefficients and offsets.
>>> + */
>>> +struct bcm2835_isp_custom_ccm {
>>> + __u32 enabled;
>>> + struct bcm2835_isp_ccm ccm;
>>> +};
>>> +
>>> +/**
>>> + * enum bcm2835_isp_gain_format - format of the gains in the lens shading
>>> + * tables used with the
>>> + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING ctrl.
>>> + *
>>> + * @GAIN_FORMAT_U0P8_1: Gains are u0.8 format, starting at 1.0
>>> + * @GAIN_FORMAT_U1P7_0: Gains are u1.7 format, starting at 0.0
>>> + * @GAIN_FORMAT_U1P7_1: Gains are u1.7 format, starting at 1.0
>>> + * @GAIN_FORMAT_U2P6_0: Gains are u2.6 format, starting at 0.0
>>> + * @GAIN_FORMAT_U2P6_1: Gains are u2.6 format, starting at 1.0
>>> + * @GAIN_FORMAT_U3P5_0: Gains are u3.5 format, starting at 0.0
>>> + * @GAIN_FORMAT_U3P5_1: Gains are u3.5 format, starting at 1.0
>>> + * @GAIN_FORMAT_U4P10: Gains are u4.10 format, starting at 0.0
>>> + */
>>> +enum bcm2835_isp_gain_format {
>>> + GAIN_FORMAT_U0P8_1 = 0,
>>> + GAIN_FORMAT_U1P7_0 = 1,
>>> + GAIN_FORMAT_U1P7_1 = 2,
>>> + GAIN_FORMAT_U2P6_0 = 3,
>>> + GAIN_FORMAT_U2P6_1 = 4,
>>> + GAIN_FORMAT_U3P5_0 = 5,
>>> + GAIN_FORMAT_U3P5_1 = 6,
>>> + GAIN_FORMAT_U4P10 = 7,
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_lens_shading - Lens shading tables supplied with the
>>> + * V4L2_CID_USER_BCM2835_ISP_LENS_SHADING
>>> + * ctrl.
>>> + *
>>> + * @enabled: Enable lens shading.
>>> + * @grid_cell_size: Size of grid cells in samples (16, 32, 64, 128 or 256).
>>> + * @grid_width: Width of lens shading tables in grid cells.
>>> + * @grid_stride: Row to row distance (in grid cells) between grid cells
>>> + * in the same horizontal location.
>>> + * @grid_height: Height of lens shading tables in grid cells.
>>> + * @mem_handle_table: Memory handle to the tables.
>>
>> What sort of handle is this? I.e. where does it come from?
>
> I believe there was a separte discusion about this. This is a vcsm
> handle that is used to store the table coefficients in memory that is
> accessible from Videocore. There is work in progress to update this
> to use dmabuf handles.
>
>>
>>> + * @ref_transform: Reference transform - unsupported, please pass zero.
>>> + * @corner_sampled: Whether the gains are sampled at the corner points
>>> + * of the grid cells or in the cell centres.
>>> + * @gain_format: Format of the gains (see enum &bcm2835_isp_gain_format).
>>> + */
>>> +struct bcm2835_isp_lens_shading {
>>> + __u32 enabled;
>>> + __u32 grid_cell_size;
>>> + __u32 grid_width;
>>> + __u32 grid_stride;
>>> + __u32 grid_height;
>>> + __u32 mem_handle_table;
>>> + __u32 ref_transform;
>>> + __u32 corner_sampled;
>>> + __u32 gain_format;
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_black_level - Sensor black level set with the
>>> + * V4L2_CID_USER_BCM2835_ISP_BLACK_LEVEL ctrl.
>>> + *
>>> + * @enabled: Enable black level.
>>> + * @black_level_r: Black level for red channel.
>>> + * @black_level_g: Black level for green channels.
>>> + * @black_level_b: Black level for blue channel.
>>> + */
>>> +struct bcm2835_isp_black_level {
>>> + __u32 enabled;
>>> + __u16 black_level_r;
>>> + __u16 black_level_g;
>>> + __u16 black_level_b;
>>> + __u8 pad_[2]; /* Unused */
>>
>> I prefer 'padding' over 'pad_'.
>
> Ack
>
>>
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_geq - Green equalisation parameters set with the
>>> + * V4L2_CID_USER_BCM2835_ISP_GEQ ctrl.
>>> + *
>>> + * @enabled: Enable green equalisation.
>>> + * @offset: Fixed offset of the green equalisation threshold.
>>> + * @slope: Slope of the green equalisation threshold.
>>> + */
>>> +struct bcm2835_isp_geq {
>>> + __u32 enabled;
>>> + __u32 offset;
>>> + struct bcm2835_isp_rational slope;
>>> +};
>>> +
>>> +#define BCM2835_NUM_GAMMA_PTS 33
>>> +
>>> +/**
>>> + * struct bcm2835_isp_gamma - Gamma parameters set with the
>>> + * V4L2_CID_USER_BCM2835_ISP_GAMMA ctrl.
>>> + *
>>> + * @enabled: Enable gamma adjustment.
>>> + * @X: X values of the points defining the gamma curve.
>>> + * Values should be scaled to 16 bits.
>>> + * @Y: Y values of the points defining the gamma curve.
>>> + * Values should be scaled to 16 bits.
>>
>> I assume 0 == black and 0xffff == white (or max luminance)?
>>
>> And so typically x[0] == y[0] == 0 and x[32] == y[32] == 0xffff?
>>
>
> Typically yes, but it is not strictily true. We could have parameters
> that clip the signal above min (0) and below max (0xffff).
>
>>> + */
>>> +struct bcm2835_isp_gamma {
>>> + __u32 enabled;
>>> + __u16 x[BCM2835_NUM_GAMMA_PTS];
>>> + __u16 y[BCM2835_NUM_GAMMA_PTS];
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_denoise - Denoise parameters set with the
>>> + * V4L2_CID_USER_BCM2835_ISP_DENOISE ctrl.
>>> + *
>>> + * @enabled: Enable denoise.
>>> + * @constant: Fixed offset of the noise threshold.
>>> + * @slope: Slope of the noise threshold.
>>> + * @strength: Denoise strength between 0.0 (off) and 1.0 (maximum).
>>> + */
>>> +struct bcm2835_isp_denoise {
>>> + __u32 enabled;
>>> + __u32 constant;
>>> + struct bcm2835_isp_rational slope;
>>> + struct bcm2835_isp_rational strength;
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_sharpen - Sharpen parameters set with the
>>> + * V4L2_CID_USER_BCM2835_ISP_SHARPEN ctrl.
>>> + *
>>> + * @enabled: Enable sharpening.
>>> + * @threshold: Threshold at which to start sharpening pixels.
>>> + * @strength: Strength with which pixel sharpening increases.
>>> + * @limit: Limit to the amount of sharpening applied.
>>> + */
>>> +struct bcm2835_isp_sharpen {
>>> + __u32 enabled;
>>> + struct bcm2835_isp_rational threshold;
>>> + struct bcm2835_isp_rational strength;
>>> + struct bcm2835_isp_rational limit;
>>> +};
>>> +
>>> +/**
>>> + * enum bcm2835_isp_dpc_mode - defective pixel correction (DPC) strength.
>>> + *
>>> + * @DPC_MODE_OFF: No DPC.
>>> + * @DPC_MODE_NORMAL: Normal DPC.
>>> + * @DPC_MODE_STRONG: Strong DPC.
>>> + */
>>> +enum bcm2835_isp_dpc_mode {
>>> + DPC_MODE_OFF = 0,
>>> + DPC_MODE_NORMAL = 1,
>>> + DPC_MODE_STRONG = 2,
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_dpc - Defective pixel correction (DPC) parameters set
>>> + * with the V4L2_CID_USER_BCM2835_ISP_DPC ctrl.
>>> + *
>>> + * @enabled: Enable DPC.
>>> + * @strength: DPC strength (see enum &bcm2835_isp_dpc_mode).
>>
>> Isn't DPC_MODE_OFF equal to just setting 'enabled' to false? If so,
>> wouldn't the 'strength' field be sufficient?
>
> This is a bit of a quirk of the hardware pipeline. DPC_MODE_OFF still
> keeps the block enabled, but running (mostly) as a passthrough. The
> enabled field physically switches off the block.
>
>>
>>> + */
>>> +struct bcm2835_isp_dpc {
>>> + __u32 enabled;
>>> + __u32 strength;
>>> +};
>>> +
>>> +/*
>>> + * ISP statistics structures.
>>> + *
>>> + * The bcm2835_isp_stats structure is generated at the output of the
>>> + * statistics node. Note that this does not directly map onto the statistics
>>> + * output of the ISP HW. Instead, the MMAL firmware code maps the HW statistics
>>> + * to the bcm2835_isp_stats structure.
>>> + */
>>> +#define DEFAULT_AWB_REGIONS_X 16
>>> +#define DEFAULT_AWB_REGIONS_Y 12
>>> +
>>> +#define NUM_HISTOGRAMS 2
>>> +#define NUM_HISTOGRAM_BINS 128
>>> +#define AWB_REGIONS (DEFAULT_AWB_REGIONS_X * DEFAULT_AWB_REGIONS_Y)
>>> +#define FLOATING_REGIONS 16
>>> +#define AGC_REGIONS 16
>>> +#define FOCUS_REGIONS 12
>>> +
>>> +/**
>>> + * struct bcm2835_isp_stats_hist - Histogram statistics
>>> + *
>>> + * @r_hist: Red channel histogram.
>>> + * @g_hist: Combined green channel histogram.
>>> + * @b_hist: Blue channel histogram.
>>> + */
>>> +struct bcm2835_isp_stats_hist {
>>> + __u32 r_hist[NUM_HISTOGRAM_BINS];
>>> + __u32 g_hist[NUM_HISTOGRAM_BINS];
>>> + __u32 b_hist[NUM_HISTOGRAM_BINS];
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_stats_region - Region sums.
>>> + *
>>> + * @counted: The number of 2x2 bayer tiles accumulated.
>>> + * @notcounted: The number of 2x2 bayer tiles not accumulated.
>>> + * @r_sum: Total sum of counted pixels in the red channel for a region.
>>> + * @g_sum: Total sum of counted pixels in the green channel for a region.
>>> + * @b_sum: Total sum of counted pixels in the blue channel for a region.
>>> + */
>>> +struct bcm2835_isp_stats_region {
>>> + __u32 counted;
>>> + __u32 notcounted;
>>> + __u64 r_sum;
>>> + __u64 g_sum;
>>> + __u64 b_sum;
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_stats_focus - Focus statistics.
>>> + *
>>> + * @contrast_val: Focus measure - accumulated output of the focus filter.
>>> + * In the first dimension, index [0] counts pixels below a
>>> + * preset threshold, and index [1] counts pixels above the
>>> + * threshold. In the second dimension, index [0] uses the
>>> + * first predefined filter, and index [1] uses the second
>>> + * predefined filter.
>>> + * @contrast_val_num: The number of counted pixels in the above accumulation.
>>> + */
>>> +struct bcm2835_isp_stats_focus {
>>> + __u64 contrast_val[2][2];
>>> + __u32 contrast_val_num[2][2];
>>> +};
>>> +
>>> +/**
>>> + * struct bcm2835_isp_stats - ISP statistics.
>>> + *
>>> + * @version: Version of the bcm2835_isp_stats structure.
>>> + * @size: Size of the bcm2835_isp_stats structure.
>>> + * @hist: Histogram statistics for the entire image.
>>> + * @awb_stats: Statistics for the regions defined for AWB calculations.
>>> + * @floating_stats: Statistics for arbitrarily placed (floating) regions.
>>> + * @agc_stats: Statistics for the regions defined for AGC calculations.
>>> + * @focus_stats: Focus filter statistics for the focus regions.
>>> + */
>>> +struct bcm2835_isp_stats {
>>> + __u32 version;
>>> + __u32 size;
>>> + struct bcm2835_isp_stats_hist hist[NUM_HISTOGRAMS];
>>> + struct bcm2835_isp_stats_region awb_stats[AWB_REGIONS];
>>> + struct bcm2835_isp_stats_region floating_stats[FLOATING_REGIONS];
>>> + struct bcm2835_isp_stats_region agc_stats[AGC_REGIONS];
>>> + struct bcm2835_isp_stats_focus focus_stats[FOCUS_REGIONS];
>>> +};
>>> +
>>> +#endif /* __BCM2835_ISP_H_ */
>>> diff --git a/drivers/staging/vc04_services/vchiq-mmal/Kconfig b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
>>> index 106f71e709df..072f3c755a68 100644
>>> --- a/drivers/staging/vc04_services/vchiq-mmal/Kconfig
>>> +++ b/drivers/staging/vc04_services/vchiq-mmal/Kconfig
>>> @@ -5,4 +5,5 @@ config BCM2835_VCHIQ_MMAL
>>> help
>>> Enables the MMAL API over VCHIQ interface as used for the
>>> majority of the multimedia services on VideoCore.
>>> - Defaults to Y when the Broadcomd BCM2835 camera host is selected.
>>> + Defaults to Y when the Broadcomd BCM2835 camera host or ISP are
>>> + selected.
>>> diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
>>> index 44ba91aa6d47..8d904fcce388 100644
>>> --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
>>> +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-encodings.h
>>> @@ -100,6 +100,10 @@
>>> */
>>> #define MMAL_ENCODING_EGL_IMAGE MMAL_FOURCC('E', 'G', 'L', 'I')
>>>
>>> +/** ISP image statistics format
>>> + */
>>> +#define MMAL_ENCODING_BRCM_STATS MMAL_FOURCC('S', 'T', 'A', 'T')
>>> +
>>> /* }@ */
>>>
>>> /** \name Pre-defined audio encodings */
>>> diff --git a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
>>> index 1793103b18fd..b3552af5cf8f 100644
>>> --- a/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
>>> +++ b/drivers/staging/vc04_services/vchiq-mmal/mmal-parameters.h
>>> @@ -221,6 +221,62 @@ enum mmal_parameter_camera_type {
>>> MMAL_PARAMETER_SHUTTER_SPEED,
>>> /**< Takes a @ref MMAL_PARAMETER_AWB_GAINS_T */
>>> MMAL_PARAMETER_CUSTOM_AWB_GAINS,
>>> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_SETTINGS_T */
>>> + MMAL_PARAMETER_CAMERA_SETTINGS,
>>> + /**< Takes a @ref MMAL_PARAMETER_PRIVACY_INDICATOR_T */
>>> + MMAL_PARAMETER_PRIVACY_INDICATOR,
>>> + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
>>> + MMAL_PARAMETER_VIDEO_DENOISE,
>>> + /**< Takes a @ref MMAL_PARAMETER_BOOLEAN_T */
>>> + MMAL_PARAMETER_STILLS_DENOISE,
>>> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_ANNOTATE_T */
>>> + MMAL_PARAMETER_ANNOTATE,
>>> + /**< Takes a @ref MMAL_PARAMETER_STEREOSCOPIC_MODE_T */
>>> + MMAL_PARAMETER_STEREOSCOPIC_MODE,
>>> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_INTERFACE_T */
>>> + MMAL_PARAMETER_CAMERA_INTERFACE,
>>> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_CLOCKING_MODE_T */
>>> + MMAL_PARAMETER_CAMERA_CLOCKING_MODE,
>>> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_CONFIG_T */
>>> + MMAL_PARAMETER_CAMERA_RX_CONFIG,
>>> + /**< Takes a @ref MMAL_PARAMETER_CAMERA_RX_TIMING_T */
>>> + MMAL_PARAMETER_CAMERA_RX_TIMING,
>>> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
>>> + MMAL_PARAMETER_DPF_CONFIG,
>>> +
>>> + /* 0x50 */
>>> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
>>> + MMAL_PARAMETER_JPEG_RESTART_INTERVAL,
>>> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
>>> + MMAL_PARAMETER_CAMERA_ISP_BLOCK_OVERRIDE,
>>> + /**< Takes a @ref MMAL_PARAMETER_LENS_SHADING_T */
>>> + MMAL_PARAMETER_LENS_SHADING_OVERRIDE,
>>> + /**< Takes a @ref MMAL_PARAMETER_UINT32_T */
>>> + MMAL_PARAMETER_BLACK_LEVEL,
>>> + /**< Takes a @ref MMAL_PARAMETER_RESIZE_T */
>>> + MMAL_PARAMETER_RESIZE_PARAMS,
>>> + /**< Takes a @ref MMAL_PARAMETER_CROP_T */
>>> + MMAL_PARAMETER_CROP,
>>> + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
>>> + MMAL_PARAMETER_OUTPUT_SHIFT,
>>> + /**< Takes a @ref MMAL_PARAMETER_INT32_T */
>>> + MMAL_PARAMETER_CCM_SHIFT,
>>> + /**< Takes a @ref MMAL_PARAMETER_CUSTOM_CCM_T */
>>> + MMAL_PARAMETER_CUSTOM_CCM,
>>> + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
>>> + MMAL_PARAMETER_ANALOG_GAIN,
>>> + /**< Takes a @ref MMAL_PARAMETER_RATIONAL_T */
>>> + MMAL_PARAMETER_DIGITAL_GAIN,
>>> + /**< Takes a @ref MMAL_PARAMETER_DENOISE_T */
>>> + MMAL_PARAMETER_DENOISE,
>>> + /**< Takes a @ref MMAL_PARAMETER_SHARPEN_T */
>>> + MMAL_PARAMETER_SHARPEN,
>>> + /**< Takes a @ref MMAL_PARAMETER_GEQ_T */
>>> + MMAL_PARAMETER_GEQ,
>>> + /**< Tales a @ref MMAP_PARAMETER_DPC_T */
>>> + MMAL_PARAMETER_DPC,
>>> + /**< Tales a @ref MMAP_PARAMETER_GAMMA_T */
>>> + MMAL_PARAMETER_GAMMA,
>>> };
>>>
>>> struct mmal_parameter_rational {
>>> @@ -779,7 +835,102 @@ struct mmal_parameter_camera_info {
>>> struct mmal_parameter_camera_info_camera
>>> cameras[MMAL_PARAMETER_CAMERA_INFO_MAX_CAMERAS];
>>> struct mmal_parameter_camera_info_flash
>>> - flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
>>> + flashes[MMAL_PARAMETER_CAMERA_INFO_MAX_FLASHES];
>>> +};
>>> +
>>> +struct mmal_parameter_ccm {
>>> + struct mmal_parameter_rational ccm[3][3];
>>> + s32 offsets[3];
>>> +};
>>> +
>>> +struct mmal_parameter_custom_ccm {
>>> + u32 enabled; /**< Enable the custom CCM. */
>>> + struct mmal_parameter_ccm ccm; /**< CCM to be used. */
>>> +};
>>> +
>>> +struct mmal_parameter_lens_shading {
>>> + u32 enabled;
>>> + u32 grid_cell_size;
>>> + u32 grid_width;
>>> + u32 grid_stride;
>>> + u32 grid_height;
>>> + u32 mem_handle_table;
>>> + u32 ref_transform;
>>> +};
>>> +
>>> +enum mmal_parameter_ls_gain_format_type {
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U0P8_1 = 0,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_0 = 1,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U1P7_1 = 2,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_0 = 3,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U2P6_1 = 4,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_0 = 5,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U3P5_1 = 6,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_U4P10 = 7,
>>> + MMAL_PARAMETER_LS_GAIN_FORMAT_TYPE_DUMMY = 0x7FFFFFFF
>>> +};
>>> +
>>> +struct mmal_parameter_lens_shading_v2 {
>>> + u32 enabled;
>>> + u32 grid_cell_size;
>>> + u32 grid_width;
>>> + u32 grid_stride;
>>> + u32 grid_height;
>>> + u32 mem_handle_table;
>>> + u32 ref_transform;
>>> + u32 corner_sampled;
>>> + enum mmal_parameter_ls_gain_format_type gain_format;
>>> +};
>>> +
>>> +struct mmal_parameter_black_level {
>>> + u32 enabled;
>>> + u16 black_level_r;
>>> + u16 black_level_g;
>>> + u16 black_level_b;
>>> + u8 pad_[2]; /* Unused */
>>> +};
>>> +
>>> +struct mmal_parameter_geq {
>>> + u32 enabled;
>>> + u32 offset;
>>> + struct mmal_parameter_rational slope;
>>> +};
>>> +
>>> +#define MMAL_NUM_GAMMA_PTS 33
>>> +struct mmal_parameter_gamma {
>>> + u32 enabled;
>>> + u16 x[MMAL_NUM_GAMMA_PTS];
>>> + u16 y[MMAL_NUM_GAMMA_PTS];
>>> +};
>>> +
>>> +struct mmal_parameter_denoise {
>>> + u32 enabled;
>>> + u32 constant;
>>> + struct mmal_parameter_rational slope;
>>> + struct mmal_parameter_rational strength;
>>> +};
>>> +
>>> +struct mmal_parameter_sharpen {
>>> + u32 enabled;
>>> + struct mmal_parameter_rational threshold;
>>> + struct mmal_parameter_rational strength;
>>> + struct mmal_parameter_rational limit;
>>> +};
>>> +
>>> +enum mmal_dpc_mode {
>>> + MMAL_DPC_MODE_OFF = 0,
>>> + MMAL_DPC_MODE_NORMAL = 1,
>>> + MMAL_DPC_MODE_STRONG = 2,
>>> + MMAL_DPC_MODE_MAX = 0x7FFFFFFF,
>>> +};
>>> +
>>> +struct mmal_parameter_dpc {
>>> + u32 enabled;
>>> + u32 strength;
>>> +};
>>> +
>>> +struct mmal_parameter_crop {
>>> + struct vchiq_mmal_rect rect;
>>> };
>>>
>>> #endif
>>>
>>
>> Regards,
>>
>> Hans
>
> Regards,
> Naush
>
^ permalink raw reply [flat|nested] 111+ messages in thread
* [PATCH v2 34/34] staging: vchiq: Load bcm2835_isp driver from vchiq
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (32 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 33/34] staging: bcm2835-isp: Add support for BC2835 ISP Laurent Pinchart
@ 2020-05-04 9:26 ` Laurent Pinchart
2020-05-04 15:15 ` [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Nicolas Saenz Julienne
34 siblings, 0 replies; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 9:26 UTC (permalink / raw)
To: linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
From: Naushir Patuck <naush@raspberrypi.com>
bcmn2835_isp is a platform driver dependent on vchiq,
therefore add the load/unload functions for it to vchiq.
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index 59ae2d479ef4..abc30e593b67 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -108,6 +108,7 @@ static DEFINE_SPINLOCK(msg_queue_spinlock);
static struct platform_device *bcm2835_camera;
static struct platform_device *bcm2835_audio;
static struct platform_device *vcsm_cma;
+static struct platform_device *bcm2835_isp;
static struct vchiq_drvdata bcm2835_drvdata = {
.cache_line_size = 32,
@@ -2824,6 +2825,7 @@ static int vchiq_probe(struct platform_device *pdev)
vcsm_cma = vchiq_register_child(pdev, "vcsm-cma");
bcm2835_camera = vchiq_register_child(pdev, "bcm2835-camera");
bcm2835_audio = vchiq_register_child(pdev, "bcm2835_audio");
+ bcm2835_isp = vchiq_register_child(pdev, "bcm2835-isp");
return 0;
@@ -2836,6 +2838,7 @@ static int vchiq_probe(struct platform_device *pdev)
static int vchiq_remove(struct platform_device *pdev)
{
+ platform_device_unregister(bcm2835_isp);
platform_device_unregister(bcm2835_audio);
platform_device_unregister(bcm2835_camera);
platform_device_unregister(vcsm_cma);
--
Regards,
Laurent Pinchart
^ permalink raw reply related [flat|nested] 111+ messages in thread* Re: [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP
2020-05-04 9:25 [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Laurent Pinchart
` (33 preceding siblings ...)
2020-05-04 9:26 ` [PATCH v2 34/34] staging: vchiq: Load bcm2835_isp driver from vchiq Laurent Pinchart
@ 2020-05-04 15:15 ` Nicolas Saenz Julienne
2020-05-04 15:38 ` Laurent Pinchart
34 siblings, 1 reply; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-04 15:15 UTC (permalink / raw)
To: Laurent Pinchart, linux-media
Cc: Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 44245 bytes --]
On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> Hello,
>
Hi Laurent, thanks for the series!
Would you mind CCing on further iterations, I was lucky enough to find this
referenced in twitter, and I'll be more than happy to review/test it. I'm the
BCM2711/BCM2835 arm architecture & vc04_services maintainer :).
An please also CC:
bcm-kernel-feedback-list@broadcom.com
linux-rpi-kernel@lists.infradead.org
linux-arm-kernel@lists.infradead.org
Regards,
Nicolas
> This patch series adds drivers for the CSI-2/CCP2 receiver and ISP found
> in the Broadcom BCM283x and compatible SoCs (namely the BCM2711). The
> CSI-2/CCP2 receiver IP core is known as Unicam. The most well known
> platforms representative of these SoCs are the Raspberry Pi. The
> previous version of the series was titled "[PATCH 0/5] Driver for the
> BCM283x CSI-2/CCP2 receiver" and contained the CSI-2/CCP2 receiver only.
>
> A driver already exists in drivers/staging/vc04_services/bcm2835-camera/
> to support cameras on Raspberry Pi platforms. The driver relies on the
> firmware running on the VC4 VPU to control the camera, and is thus
> limited to the camera sensors supported in the firmware. These drivers,
> on the other hand, have limited dependencies on the firmware:
>
> - The Unicam peripheral driver doesn't rely on the firmware at all and
> controls the Unicam hardware directly. It supports any camera sensor
> compatible with those interfaces.
>
> - The ISP driver relies on the VC4 VPU firmware to communicate with the
> ISP hardware, but, unlike with the existing bcm2835-camera driver, the
> firmware exposes the ISP instead of controlling it internally.
>
> The code is upported from the Raspberry Pi Linux kernel repository
> available at https://github.com/raspberrypi/linux. The patches are based
> on top of v5.7-rc2 with Jacopo's "[PATCH v4 0/5] media: Register
> read-only sub-dev devnode" series applied.
>
> Patches 01/34 to 05/34 are related to the bcm2835-unicam driver. Patches
> 01/34 and 02/34 add new a pixel format and media bus code for sensor
> embedded data. Patch 03/34 then adds DT bindings for the driver, and
> patch 04/34 adds the driver itself. Patch 05/34 adds the Unicam
> peripheral instances to the BCM2711 device tree.
>
> The bcm2835-unicam driver supports capturing both image data and
> embedded data. Support for embedded data is currently implemented
> through two sink pads, requiring two source pads on the sensor driver
> side. Work is ongoing to try and replace this with usage of a
> multiplexed streams API.
>
> The remaining patches are related to the bcm2835-isp driver. As the
> driver needs to communicate with the VC4 VPU firmware, whose support is
> currently in staging, the new driver is added to staging too.
>
> Patch 06/34 adds a new driver named vc-sm-cma to handle memory sharing
> with the VC4 VPU. This will likely be reworked in the future to use
> dmabuf heaps. Patch 07/34 then breaks the VC4 VPU multimedia access
> layer code (named vchiq-mmal) out of the existing bcm2835-camera driver
> to a new directory, to be shared with the bcm2835-isp driver. Patches
> 08/34 to 24/34 then upport various improvements to the MMAL code.
> Patches 25/34 to 31/34 follow with an upport of various improvement to
> the VCHIQ code, which handles the communication with the VC4 VPU (and is
> used by the MMAL code).
>
> At patch 32/34 we move to the media side, with a small extension to
> videobuf2. Patch 33/34 adds the bcm2835-isp driver, along with
> documentation of the related metadata format. Patch 34/34 then wires
> this up with the vchiq-mmal driver.
>
> The two drivers will likely be split into separate patch series for v3.
> I however wanted to bundle them here to emphasize that they're related,
> and that together they support full control of the Raspberry Pi cameras
> through libcamera without any closed-source software. The corresponding
> libcamera code is available from
>
> git://linuxtv.org/libcamera.git raspberrypi
>
> The 'raspberrypi' branch is temporary until the code gets integrated in
> the master branch after public review.
>
> I would like to thank Dave Stevenson, Naushir Patuk and David Plowman
> from Raspberry Pi (Trading) Ltd. for their hard work that made this all
> possible, as well as Jacopo Mondi, Kieran Bingham and Niklas Söderlund
> from the libcamera team for all their efforts on both the kernel and
> libcamera side. This is, I'm sure, the beginning of a new journey for
> computational camera support in Linux.
>
> And now, the customary v4l2-compliance report. There are three new
> failures with bcm2835-unicam compared to the previous version, and they
> will addressed in v3.
>
> pi@raspberrypi:~/src/v4l-utils $ ./utils/v4l2-compliance/v4l2-compliance -m
> /dev/media0
> v4l2-compliance SHA: 2984938795a23e4bdf5a4b75c12a4245a2e0daff, 32 bits, 32-
bit
> time_t
>
> Compliance test for unicam device /dev/media0:
>
> Media Driver Info:
> Driver name : unicam
> Model : unicam
> Serial :
> Bus info : platform:fe801000.csi
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
>
> Required ioctls:
> test MEDIA_IOC_DEVICE_INFO: OK
>
> Allow for multiple opens:
> test second /dev/media0 open: OK
> test MEDIA_IOC_DEVICE_INFO: OK
> test for unlimited opens: OK
>
> Media Controller ioctls:
> test MEDIA_IOC_G_TOPOLOGY: OK
> Entities: 3 Interfaces: 3 Pads: 4 Links: 5
> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> test MEDIA_IOC_SETUP_LINK: OK
> test invalid ioctls: OK
>
> Total for unicam device /dev/media0: 8, Succeeded: 8, Failed: 0, Warnings: 0
> ------------------------------------------------------------------------------
> --
> Compliance test for unicam device /dev/video0:
>
> Driver Info:
> Driver name : unicam
> Card type : unicam
> Bus info : platform:fe801000.csi
> Driver version : 5.7.0
> Capabilities : 0x84a00001
> Video Capture
> Metadata Capture
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04200001
> Video Capture
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : unicam
> Model : unicam
> Serial :
> Bus info : platform:fe801000.csi
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
> Interface Info:
> ID : 0x03000006
> Type : V4L Video
> Entity Info:
> ID : 0x00000004 (4)
> Name : unicam-image
> Function : V4L2 I/O
> Flags : default
> Pad 0x01000005 : 0: Sink
> Link 0x02000008: from remote pad 0x1000002 of entity 'imx219 10-
> 0010': Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video0 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> test invalid ioctls: OK
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 1 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls (Input 0):
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 16 Private Controls: 0
>
> Format ioctls (Input 0):
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> fail: v4l2-test-formats.cpp(1038): Global format mismatch:
> 41415270(pRAA)/640x480 vs 30314752(RG10)/640x480
> test VIDIOC_S_FMT: FAIL
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> fail: v4l2-test-formats.cpp(1529): !doioctl(node,
> VIDIOC_G_SELECTION, &sel)
> fail: v4l2-test-formats.cpp(1547): testLegacyCrop(node)
> test Cropping: FAIL
> test Composing: OK (Not Supported)
> test Scaling: OK (Not Supported)
>
> Codec ioctls (Input 0):
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls (Input 0):
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Total for unicam device /dev/video0: 46, Succeeded: 44, Failed: 2, Warnings: 0
> ------------------------------------------------------------------------------
> --
> Compliance test for unicam device /dev/video1:
>
> Driver Info:
> Driver name : unicam
> Card type : unicam
> Bus info : platform:fe801000.csi
> Driver version : 5.7.0
> Capabilities : 0x84a00001
> Video Capture
> Metadata Capture
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04a00000
> Metadata Capture
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : unicam
> Model : unicam
> Serial :
> Bus info : platform:fe801000.csi
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
> Interface Info:
> ID : 0x0300000c
> Type : V4L Video
> Entity Info:
> ID : 0x0000000a (10)
> Name : unicam-embedded
> Function : V4L2 I/O
> Pad 0x0100000b : 0: Sink
> Link 0x0200000e: from remote pad 0x1000003 of entity 'imx219 10-
> 0010': Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video1 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> test invalid ioctls: OK
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 1 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls (Input 0):
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls (Input 0):
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK (Not Supported)
>
> Codec ioctls (Input 0):
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls (Input 0):
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Total for unicam device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0
> ------------------------------------------------------------------------------
> --
> Compliance test for device /dev/v4l-subdev0:
>
>
> Required ioctls:
>
> Allow for multiple opens:
> test second /dev/v4l-subdev0 open: OK
> test for unlimited opens: OK
>
> test invalid ioctls: OK
> Debug ioctls:
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 16 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK (Not Supported)
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK (Not Supported)
> test VIDIOC_TRY_FMT: OK (Not Supported)
> test VIDIOC_S_FMT: OK (Not Supported)
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK (Not Supported)
> test VIDIOC_EXPBUF: OK (Not Supported)
> test Requests: OK (Not Supported)
>
> Total for device /dev/v4l-subdev0: 41, Succeeded: 41, Failed: 0, Warnings: 0
>
> Grand Total for unicam device /dev/media0: 141, Succeeded: 139, Failed: 2,
> Warnings: 0
> pi@raspberrypi:~/src/v4l-utils $ ./utils/v4l2-compliance/v4l2-compliance -m
> /dev/media1
> v4l2-compliance SHA: 2984938795a23e4bdf5a4b75c12a4245a2e0daff, 32 bits, 32-
bit
> time_t
>
> Compliance test for bcm2835-isp device /dev/media1:
>
> Media Driver Info:
> Driver name : bcm2835-isp
> Model : bcm2835-isp
> Serial :
> Bus info : platform:bcm2835-isp
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
>
> Required ioctls:
> test MEDIA_IOC_DEVICE_INFO: OK
>
> Allow for multiple opens:
> test second /dev/media1 open: OK
> test MEDIA_IOC_DEVICE_INFO: OK
> test for unlimited opens: OK
>
> Media Controller ioctls:
> test MEDIA_IOC_G_TOPOLOGY: OK
> Entities: 5 Interfaces: 4 Pads: 8 Links: 8
> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
> test MEDIA_IOC_SETUP_LINK: OK
> test invalid ioctls: OK
>
> Total for bcm2835-isp device /dev/media1: 8, Succeeded: 8, Failed: 0,
> Warnings: 0
> ------------------------------------------------------------------------------
> --
> Compliance test for bcm2835-isp device /dev/video13:
>
> Driver Info:
> Driver name : bcm2835-isp
> Card type : bcm2835-isp
> Bus info : platform:bcm2835-isp
> Driver version : 5.7.0
> Capabilities : 0x84200002
> Video Output
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04200002
> Video Output
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : bcm2835-isp
> Model : bcm2835-isp
> Serial :
> Bus info : platform:bcm2835-isp
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
> Interface Info:
> ID : 0x03000008
> Type : V4L Video
> Entity Info:
> ID : 0x00000006 (6)
> Name : bcm2835-isp0-output0
> Function : V4L2 I/O
> Pad 0x01000007 : 0: Source
> Link 0x0200000a: to remote pad 0x1000002 of entity 'bcm2835_isp0':
> Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video13 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> test invalid ioctls: OK
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
> test VIDIOC_QUERYCTRL: OK
> test VIDIOC_G/S_CTRL: OK
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 5 Private Controls: 8
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Total for bcm2835-isp device /dev/video13: 46, Succeeded: 46, Failed: 0,
> Warnings: 0
> ------------------------------------------------------------------------------
> --
> Compliance test for bcm2835-isp device /dev/video14:
>
> Driver Info:
> Driver name : bcm2835-isp
> Card type : bcm2835-isp
> Bus info : platform:bcm2835-isp
> Driver version : 5.7.0
> Capabilities : 0x84200001
> Video Capture
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04200001
> Video Capture
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : bcm2835-isp
> Model : bcm2835-isp
> Serial :
> Bus info : platform:bcm2835-isp
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
> Interface Info:
> ID : 0x0300000e
> Type : V4L Video
> Entity Info:
> ID : 0x0000000c (12)
> Name : bcm2835-isp0-capture1
> Function : V4L2 I/O
> Pad 0x0100000d : 0: Sink
> Link 0x02000010: from remote pad 0x1000003 of entity 'bcm2835_isp0':
> Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video14 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> test invalid ioctls: OK
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Total for bcm2835-isp device /dev/video14: 46, Succeeded: 46, Failed: 0,
> Warnings: 0
> ------------------------------------------------------------------------------
> --
> Compliance test for bcm2835-isp device /dev/video15:
>
> Driver Info:
> Driver name : bcm2835-isp
> Card type : bcm2835-isp
> Bus info : platform:bcm2835-isp
> Driver version : 5.7.0
> Capabilities : 0x84200001
> Video Capture
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04200001
> Video Capture
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : bcm2835-isp
> Model : bcm2835-isp
> Serial :
> Bus info : platform:bcm2835-isp
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
> Interface Info:
> ID : 0x03000014
> Type : V4L Video
> Entity Info:
> ID : 0x00000012 (18)
> Name : bcm2835-isp0-capture2
> Function : V4L2 I/O
> Pad 0x01000013 : 0: Sink
> Link 0x02000016: from remote pad 0x1000004 of entity 'bcm2835_isp0':
> Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video15 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> test invalid ioctls: OK
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Total for bcm2835-isp device /dev/video15: 46, Succeeded: 46, Failed: 0,
> Warnings: 0
> ------------------------------------------------------------------------------
> --
> Compliance test for bcm2835-isp device /dev/video16:
>
> Driver Info:
> Driver name : bcm2835-isp
> Card type : bcm2835-isp
> Bus info : platform:bcm2835-isp
> Driver version : 5.7.0
> Capabilities : 0x84a00000
> Metadata Capture
> Streaming
> Extended Pix Format
> Device Capabilities
> Device Caps : 0x04a00000
> Metadata Capture
> Streaming
> Extended Pix Format
> Media Driver Info:
> Driver name : bcm2835-isp
> Model : bcm2835-isp
> Serial :
> Bus info : platform:bcm2835-isp
> Media version : 5.7.0
> Hardware revision: 0x00000000 (0)
> Driver version : 5.7.0
> Interface Info:
> ID : 0x0300001a
> Type : V4L Video
> Entity Info:
> ID : 0x00000018 (24)
> Name : bcm2835-isp0-capture3
> Function : V4L2 I/O
> Pad 0x01000019 : 0: Sink
> Link 0x0200001c: from remote pad 0x1000005 of entity 'bcm2835_isp0':
> Data, Enabled, Immutable
>
> Required ioctls:
> test MC information (see 'Media Driver Info' above): OK
> test VIDIOC_QUERYCAP: OK
>
> Allow for multiple opens:
> test second /dev/video16 open: OK
> test VIDIOC_QUERYCAP: OK
> test VIDIOC_G/S_PRIORITY: OK
> test for unlimited opens: OK
>
> test invalid ioctls: OK
> Debug ioctls:
> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
> test VIDIOC_LOG_STATUS: OK (Not Supported)
>
> Input ioctls:
> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
> test VIDIOC_ENUMAUDIO: OK (Not Supported)
> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDIO: OK (Not Supported)
> Inputs: 0 Audio Inputs: 0 Tuners: 0
>
> Output ioctls:
> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
> Outputs: 0 Audio Outputs: 0 Modulators: 0
>
> Input/Output configuration ioctls:
> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
> test VIDIOC_G/S_EDID: OK (Not Supported)
>
> Control ioctls:
> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
> test VIDIOC_QUERYCTRL: OK (Not Supported)
> test VIDIOC_G/S_CTRL: OK (Not Supported)
> test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
> Standard Controls: 0 Private Controls: 0
>
> Format ioctls:
> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
> test VIDIOC_G/S_PARM: OK (Not Supported)
> test VIDIOC_G_FBUF: OK (Not Supported)
> test VIDIOC_G_FMT: OK
> test VIDIOC_TRY_FMT: OK
> test VIDIOC_S_FMT: OK
> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
> test Cropping: OK (Not Supported)
> test Composing: OK (Not Supported)
> test Scaling: OK (Not Supported)
>
> Codec ioctls:
> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>
> Buffer ioctls:
> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
> test VIDIOC_EXPBUF: OK
> test Requests: OK (Not Supported)
>
> Total for bcm2835-isp device /dev/video16: 46, Succeeded: 46, Failed: 0,
> Warnings: 0
>
> Grand Total for bcm2835-isp device /dev/media1: 192, Succeeded: 192, Failed:
> 0, Warnings: 0
>
> Dave Stevenson (22):
> dt-bindings: media: Document BCM283x CSI2/CCP2 receiver
> staging: vc04_services: Add new vc-sm-cma driver
> staging: mmal-vchiq: Allocate and free components as required
> staging: mmal-vchiq: Avoid use of bool in structures
> staging: mmal-vchiq: Make timeout a defined parameter
> staging: mmal-vchiq: Make a mmal_buf struct for passing parameters
> staging: mmal-vchiq: Add support for event callbacks.
> staging: mmal-vchiq: Support sending data to MMAL ports
> staging: mmal-vchiq: Fixup vchiq-mmal include ordering
> staging: mmal-vchiq: Use vc-sm-cma to support zero copy
> staging: mmal-vchiq: Fix client_component for 64 bit kernel
> staging: mmal_vchiq: Add in the Bayer encoding formats
> staging: mmal-vchiq: Always return the param size from param_get
> staging: mmal-vchiq: If the VPU returns an error, don't negate it
> staging: mmal-vchiq: Fix handling of VB2_MEMORY_DMABUF buffers
> staging: mmal-vchiq: Update mmal_parameters.h with recently defined
> params
> staging: mmal-vchiq: Free the event context for control ports
> staging: mmal-vchiq: Fix memory leak in error path
> staging: vchiq_arm: Register vcsm-cma as a platform driver
> staging: vchiq_arm: Set up dma ranges on child devices
> staging: vchiq: Use the old dma controller for OF config on platform
> devices
> media: videobuf2: Allow exporting of a struct dmabuf
>
> Jacopo Mondi (1):
> staging: bcm2835: Break MMAL support out from camera
>
> Laurent Pinchart (1):
> ARM: dts: bcm2711: Add Unicam DT nodes
>
> Naushir Patuck (6):
> media: uapi: v4l2-core: Add sensor ancillary data V4L2 fourcc type
> media: uapi: Add MEDIA_BUS_FMT_SENSOR_DATA media bus format
> media: bcm2835-unicam: Driver for CCP2/CSI2 camera interface
> staging: mmal-vchiq: Fix formatting errors in mmal_parameters.h
> staging: bcm2835-isp: Add support for BC2835 ISP
> staging: vchiq: Load bcm2835_isp driver from vchiq
>
> Oliver Gjoneski (1):
> staging: vchiq_2835_arm: Implement a DMA pool for small bulk transfers
>
> Phil Elwell (3):
> staging: vchiq: Add 36-bit address support
> staging: vchiq_arm: Give vchiq children DT nodes
> staging: vchiq_arm: Add a matching unregister call
>
> .../bindings/media/brcm,bcm2835-unicam.yaml | 155 +
> Documentation/media/uapi/v4l/meta-formats.rst | 1 +
> .../uapi/v4l/pixfmt-meta-sensor-data.rst | 32 +
> .../media/uapi/v4l/subdev-formats.rst | 33 +
> MAINTAINERS | 7 +
> arch/arm/boot/dts/bcm2711.dtsi | 43 +
> .../media/common/videobuf2/videobuf2-core.c | 21 +-
> drivers/media/platform/Kconfig | 1 +
> drivers/media/platform/Makefile | 2 +
> drivers/media/platform/bcm2835/Kconfig | 15 +
> drivers/media/platform/bcm2835/Makefile | 3 +
> .../media/platform/bcm2835/bcm2835-unicam.c | 2825 +++++++++++++++++
> .../media/platform/bcm2835/vc4-regs-unicam.h | 253 ++
> drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
> .../v4l/pixfmt-meta-bcm2835-isp-stats.rst | 41 +
> drivers/staging/vc04_services/Kconfig | 6 +
> drivers/staging/vc04_services/Makefile | 7 +-
> .../vc04_services/bcm2835-camera/Kconfig | 1 +
> .../vc04_services/bcm2835-camera/Makefile | 4 +-
> .../bcm2835-camera/bcm2835-camera.c | 64 +-
> .../staging/vc04_services/bcm2835-isp/Kconfig | 14 +
> .../vc04_services/bcm2835-isp/Makefile | 10 +
> .../bcm2835-isp/bcm2835-v4l2-isp.c | 1632 ++++++++++
> .../bcm2835-isp/bcm2835_isp_ctrls.h | 67 +
> .../bcm2835-isp/bcm2835_isp_fmts.h | 301 ++
> .../include/linux/broadcom/vc_sm_cma_ioctl.h | 114 +
> .../include/uapi/linux/bcm2835-isp.h | 333 ++
> .../interface/vchiq_arm/vchiq_2835_arm.c | 126 +-
> .../interface/vchiq_arm/vchiq_arm.c | 38 +
> .../interface/vchiq_arm/vchiq_arm.h | 1 +
> .../staging/vc04_services/vc-sm-cma/Kconfig | 10 +
> .../staging/vc04_services/vc-sm-cma/Makefile | 13 +
> drivers/staging/vc04_services/vc-sm-cma/TODO | 1 +
> .../staging/vc04_services/vc-sm-cma/vc_sm.c | 1732 ++++++++++
> .../staging/vc04_services/vc-sm-cma/vc_sm.h | 84 +
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.c | 505 +++
> .../vc04_services/vc-sm-cma/vc_sm_cma_vchi.h | 63 +
> .../vc04_services/vc-sm-cma/vc_sm_defs.h | 300 ++
> .../vc04_services/vc-sm-cma/vc_sm_knl.h | 28 +
> .../staging/vc04_services/vchiq-mmal/Kconfig | 9 +
> .../staging/vc04_services/vchiq-mmal/Makefile | 8 +
> .../mmal-common.h | 10 +
> .../mmal-encodings.h | 31 +
> .../mmal-msg-common.h | 0
> .../mmal-msg-format.h | 0
> .../mmal-msg-port.h | 0
> .../{bcm2835-camera => vchiq-mmal}/mmal-msg.h | 36 +
> .../mmal-parameters.h | 231 +-
> .../mmal-vchiq.c | 376 ++-
> .../mmal-vchiq.h | 13 +-
> include/media/videobuf2-core.h | 15 +
> include/uapi/linux/media-bus-format.h | 3 +
> include/uapi/linux/videodev2.h | 1 +
> 53 files changed, 9492 insertions(+), 128 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-
> unicam.yaml
> create mode 100644 Documentation/media/uapi/v4l/pixfmt-meta-sensor-data.rst
> create mode 100644 drivers/media/platform/bcm2835/Kconfig
> create mode 100644 drivers/media/platform/bcm2835/Makefile
> create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c
> create mode 100644 drivers/media/platform/bcm2835/vc4-regs-unicam.h
> create mode 100644 drivers/staging/vc04_services/Documentation/userspace-
> api/media/v4l/pixfmt-meta-bcm2835-isp-stats.rst
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Kconfig
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/Makefile
> create mode 100644 drivers/staging/vc04_services/bcm2835-isp/bcm2835-v4l2-
> isp.c
> create mode 100644 drivers/staging/vc04_services/bcm2835-
> isp/bcm2835_isp_ctrls.h
> create mode 100644 drivers/staging/vc04_services/bcm2835-
> isp/bcm2835_isp_fmts.h
> create mode 100644
> drivers/staging/vc04_services/include/linux/broadcom/vc_sm_cma_ioctl.h
> create mode 100644 drivers/staging/vc04_services/include/uapi/linux/bcm2835-
> isp.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Kconfig
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/Makefile
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/TODO
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.c
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_cma_vchi.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_defs.h
> create mode 100644 drivers/staging/vc04_services/vc-sm-cma/vc_sm_knl.h
> create mode 100644 drivers/staging/vc04_services/vchiq-mmal/Kconfig
> create mode 100644 drivers/staging/vc04_services/vchiq-mmal/Makefile
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-
> common.h (84%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-
> encodings.h (80%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-msg-
> common.h (100%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-msg-
> format.h (100%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-msg-
> port.h (100%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-
> msg.h (90%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-
> parameters.h (77%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-
> vchiq.c (82%)
> rename drivers/staging/vc04_services/{bcm2835-camera => vchiq-mmal}/mmal-
> vchiq.h (93%)
>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread* Re: [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP
2020-05-04 15:15 ` [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP Nicolas Saenz Julienne
@ 2020-05-04 15:38 ` Laurent Pinchart
2020-05-04 16:15 ` Nicolas Saenz Julienne
0 siblings, 1 reply; 111+ messages in thread
From: Laurent Pinchart @ 2020-05-04 15:38 UTC (permalink / raw)
To: Nicolas Saenz Julienne
Cc: linux-media, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
Hi Nicolas,
On Mon, May 04, 2020 at 05:15:42PM +0200, Nicolas Saenz Julienne wrote:
> On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > Hello,
>
> Hi Laurent, thanks for the series!
> Would you mind CCing on further iterations, I was lucky enough to find this
> referenced in twitter, and I'll be more than happy to review/test it. I'm the
> BCM2711/BCM2835 arm architecture & vc04_services maintainer :).
>
> An please also CC:
> bcm-kernel-feedback-list@broadcom.com
> linux-rpi-kernel@lists.infradead.org
> linux-arm-kernel@lists.infradead.org
Sure. I was so focussed on the userspace part and on getting everything
ready for today's release that I completely forgot to run the patches
through get_maintainers.pl and CC the appropriate people and lists.
Sorry about the oversight, I'll make sure to fix it next time.
Speaking of BCM2711/BCM2835 maintenance... :-) This series isn't
complete as it's missing full DT integration of camera sensor modules.
This is handled through DT overlays in the Raspberry Pi downstream
kernel, which are applied to the base DT by the boot loader (a neat
solution in my opinion). I wonder what we should do for mainline.
Ideally we should be able to at least use the DT overlays from the
downstream Raspberry Pi kernel on mainline, but that's not possible
today due to incompatibilities in the base DT (in particular related to
the I2C port to which the camera sensors are connected). Dave, have you
given that a thought by any chance ?
Another more general question for Nicolas, is there a plan to try and
get vc04_services out of staging ?
> > This patch series adds drivers for the CSI-2/CCP2 receiver and ISP found
> > in the Broadcom BCM283x and compatible SoCs (namely the BCM2711). The
> > CSI-2/CCP2 receiver IP core is known as Unicam. The most well known
> > platforms representative of these SoCs are the Raspberry Pi. The
> > previous version of the series was titled "[PATCH 0/5] Driver for the
> > BCM283x CSI-2/CCP2 receiver" and contained the CSI-2/CCP2 receiver only.
> >
> > A driver already exists in drivers/staging/vc04_services/bcm2835-camera/
> > to support cameras on Raspberry Pi platforms. The driver relies on the
> > firmware running on the VC4 VPU to control the camera, and is thus
> > limited to the camera sensors supported in the firmware. These drivers,
> > on the other hand, have limited dependencies on the firmware:
> >
> > - The Unicam peripheral driver doesn't rely on the firmware at all and
> > controls the Unicam hardware directly. It supports any camera sensor
> > compatible with those interfaces.
> >
> > - The ISP driver relies on the VC4 VPU firmware to communicate with the
> > ISP hardware, but, unlike with the existing bcm2835-camera driver, the
> > firmware exposes the ISP instead of controlling it internally.
> >
> > The code is upported from the Raspberry Pi Linux kernel repository
> > available at https://github.com/raspberrypi/linux. The patches are based
> > on top of v5.7-rc2 with Jacopo's "[PATCH v4 0/5] media: Register
> > read-only sub-dev devnode" series applied.
> >
> > Patches 01/34 to 05/34 are related to the bcm2835-unicam driver. Patches
> > 01/34 and 02/34 add new a pixel format and media bus code for sensor
> > embedded data. Patch 03/34 then adds DT bindings for the driver, and
> > patch 04/34 adds the driver itself. Patch 05/34 adds the Unicam
> > peripheral instances to the BCM2711 device tree.
> >
> > The bcm2835-unicam driver supports capturing both image data and
> > embedded data. Support for embedded data is currently implemented
> > through two sink pads, requiring two source pads on the sensor driver
> > side. Work is ongoing to try and replace this with usage of a
> > multiplexed streams API.
> >
> > The remaining patches are related to the bcm2835-isp driver. As the
> > driver needs to communicate with the VC4 VPU firmware, whose support is
> > currently in staging, the new driver is added to staging too.
> >
> > Patch 06/34 adds a new driver named vc-sm-cma to handle memory sharing
> > with the VC4 VPU. This will likely be reworked in the future to use
> > dmabuf heaps. Patch 07/34 then breaks the VC4 VPU multimedia access
> > layer code (named vchiq-mmal) out of the existing bcm2835-camera driver
> > to a new directory, to be shared with the bcm2835-isp driver. Patches
> > 08/34 to 24/34 then upport various improvements to the MMAL code.
> > Patches 25/34 to 31/34 follow with an upport of various improvement to
> > the VCHIQ code, which handles the communication with the VC4 VPU (and is
> > used by the MMAL code).
> >
> > At patch 32/34 we move to the media side, with a small extension to
> > videobuf2. Patch 33/34 adds the bcm2835-isp driver, along with
> > documentation of the related metadata format. Patch 34/34 then wires
> > this up with the vchiq-mmal driver.
> >
> > The two drivers will likely be split into separate patch series for v3.
> > I however wanted to bundle them here to emphasize that they're related,
> > and that together they support full control of the Raspberry Pi cameras
> > through libcamera without any closed-source software. The corresponding
> > libcamera code is available from
> >
> > git://linuxtv.org/libcamera.git raspberrypi
> >
> > The 'raspberrypi' branch is temporary until the code gets integrated in
> > the master branch after public review.
> >
> > I would like to thank Dave Stevenson, Naushir Patuk and David Plowman
> > from Raspberry Pi (Trading) Ltd. for their hard work that made this all
> > possible, as well as Jacopo Mondi, Kieran Bingham and Niklas Söderlund
> > from the libcamera team for all their efforts on both the kernel and
> > libcamera side. This is, I'm sure, the beginning of a new journey for
> > computational camera support in Linux.
> >
> > And now, the customary v4l2-compliance report. There are three new
> > failures with bcm2835-unicam compared to the previous version, and they
> > will addressed in v3.
[snip]
--
Regards,
Laurent Pinchart
^ permalink raw reply [flat|nested] 111+ messages in thread
* Re: [PATCH v2 00/34] Drivers for the BCM283x CSI-2/CCP2 receiver and ISP
2020-05-04 15:38 ` Laurent Pinchart
@ 2020-05-04 16:15 ` Nicolas Saenz Julienne
0 siblings, 0 replies; 111+ messages in thread
From: Nicolas Saenz Julienne @ 2020-05-04 16:15 UTC (permalink / raw)
To: Laurent Pinchart
Cc: linux-media, Kieran Bingham, Jacopo Mondi, Niklas Söderlund,
Naushir Patuck, Dave Stevenson
[-- Attachment #1: Type: text/plain, Size: 2724 bytes --]
On Mon, 2020-05-04 at 18:38 +0300, Laurent Pinchart wrote:
> Hi Nicolas,
>
> On Mon, May 04, 2020 at 05:15:42PM +0200, Nicolas Saenz Julienne wrote:
> > On Mon, 2020-05-04 at 12:25 +0300, Laurent Pinchart wrote:
> > > Hello,
> >
> > Hi Laurent, thanks for the series!
> > Would you mind CCing on further iterations, I was lucky enough to find this
> > referenced in twitter, and I'll be more than happy to review/test it. I'm
> > the
> > BCM2711/BCM2835 arm architecture & vc04_services maintainer :).
> >
> > An please also CC:
> > bcm-kernel-feedback-list@broadcom.com
> > linux-rpi-kernel@lists.infradead.org
> > linux-arm-kernel@lists.infradead.org
>
> Sure. I was so focussed on the userspace part and on getting everything
> ready for today's release that I completely forgot to run the patches
> through get_maintainers.pl and CC the appropriate people and lists.
> Sorry about the oversight, I'll make sure to fix it next time.
No worries!
> Speaking of BCM2711/BCM2835 maintenance... :-) This series isn't
> complete as it's missing full DT integration of camera sensor modules.
> This is handled through DT overlays in the Raspberry Pi downstream
> kernel, which are applied to the base DT by the boot loader (a neat
> solution in my opinion). I wonder what we should do for mainline.
> Ideally we should be able to at least use the DT overlays from the
> downstream Raspberry Pi kernel on mainline, but that's not possible
> today due to incompatibilities in the base DT (in particular related to
> the I2C port to which the camera sensors are connected). Dave, have you
> given that a thought by any chance ?
Could you point out the specific overlays? I'd suggest to add an example in the
bindings and/or sending a 'not to be picked-up' patch to the mailing lists
showing how it's meant to work. That way people messing with that interface can
at least find it and work from there. See here for example Eric Anholt dealing
with a similar issue as yours https://lkml.org/lkml/2016/12/14/483. I can't say
for other distributions, but in openSUSE we try to support the downstream
overlays.
BTW, is the I2C issue you mention related to this downstream patch
5bb4694516761 ('dt: Implement an I2C pinctrl mux for BSC0')?
> Another more general question for Nicolas, is there a plan to try and
> get vc04_services out of staging ?
There's noone going at it seriously. I try to make the occasional cleanup here
and there, and the occasional patch might show up. But that's the extent of it.
I'm interested in biting the bullet but I've been fairly busy.
Maybe this series can motivate people into investing more time into it :).
Regards,
Nicolas
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 111+ messages in thread