Sony imx219 driver?

Sony imx219 driver?

[Dave Stevenson]

Hi All.

Just a quick query to avoid duplicating effort. Has anyone worked on a 
Sony IMX219 (or other Sony sensor) subdevice driver as yet?

With the new Raspberry Pi camera being IMX219, and as Broadcom have 
released an soc_camera based driver for the sensor already 
(https://android.googlesource.com/kernel/bcm/+/android-bcm-tetra-3.10-lollipop-wear-release/drivers/media/video/imx219.c) 
I was going to investigate converting that to a subdevice. I just wanted 
to check this wasn't already in someone else's work queue.

A further Google shows that there's also an soc_camera IMX219 driver in 
ChromiumOS, copyright Andrew Chew @ Nvidia, but author Guennadi 
Liakhovetski who I know posts on here. 
https://chromium.googlesource.com/chromiumos/third_party/kernel/+/factory-ryu-6486.14.B-chromeos-3.14/drivers/media/i2c/soc_camera/imx219.c. 
The Broadcom one supports 8MPix and 1080P, the Chromium one only 8MP. 
Perhaps a hybrid of the feature set?
Throw in 
https://github.com/ZenfoneArea/android_kernel_asus_zenfone5/blob/master/linux/modules/camera/drivers/media/i2c/imx219/imx219.h 
as well, and we have register sets for numerous readout modes, plus 
there are the ones in the Pi firmware which can be extracted if necessary.


On a related note, if putting together a system with IMX219 or similar 
producing Bayer raw 10, the data on the CSI2 bus is one of the 
V4L2_PIX_FMT_SRGGB10P formats. What's the correct way to reflect that 
from the sensor subdevice in an MEDIA_BUS_FMT_ enum?
The closest is MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE (or LE), but the data 
isn't padded (the Pi CSI2 receiver can do the unpacking and padding, but 
that just takes up more memory).|||| Or is it MEDIA_BUS_FMT_SBGGR10_1X10 
to describe the data on the bus correctly as 10bpp Bayer, and the odd 
packing is ignored. Or do we need new enums?

Thanks.
   Dave

Re: Sony imx219 driver?

[Hans Verkuil]

On 07/21/2016 08:19 PM, Dave Stevenson wrote:
> Hi All.
> 
> Just a quick query to avoid duplicating effort. Has anyone worked on a 
> Sony IMX219 (or other Sony sensor) subdevice driver as yet?

Not that I am aware of.

> With the new Raspberry Pi camera being IMX219, and as Broadcom have 
> released an soc_camera based driver for the sensor already 
> (https://android.googlesource.com/kernel/bcm/+/android-bcm-tetra-3.10-lollipop-wear-release/drivers/media/video/imx219.c) 

#@!@$! Why do they never ever ask us what would be a good example driver to use. The
soc-camera framework is deprecated and will likely be gone by the end of the year.

> I was going to investigate converting that to a subdevice. I just wanted 
> to check this wasn't already in someone else's work queue.
> 
> A further Google shows that there's also an soc_camera IMX219 driver in 
> ChromiumOS, copyright Andrew Chew @ Nvidia, but author Guennadi 
> Liakhovetski who I know posts on here. 
> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/factory-ryu-6486.14.B-chromeos-3.14/drivers/media/i2c/soc_camera/imx219.c. 

At least for the tegra there is a proper non-soc-camera driver in the works.

> The Broadcom one supports 8MPix and 1080P, the Chromium one only 8MP. 
> Perhaps a hybrid of the feature set?

They usually only implement what they need, so I'd say for chromium they only needed 8MP support.

> Throw in 
> https://github.com/ZenfoneArea/android_kernel_asus_zenfone5/blob/master/linux/modules/camera/drivers/media/i2c/imx219/imx219.h 
> as well, and we have register sets for numerous readout modes, plus 
> there are the ones in the Pi firmware which can be extracted if necessary.

With all that code it shouldn't be hard to write a proper subdev driver.

> 
> On a related note, if putting together a system with IMX219 or similar 
> producing Bayer raw 10, the data on the CSI2 bus is one of the 
> V4L2_PIX_FMT_SRGGB10P formats. What's the correct way to reflect that 
> from the sensor subdevice in an MEDIA_BUS_FMT_ enum?
> The closest is MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE (or LE), but the data 
> isn't padded (the Pi CSI2 receiver can do the unpacking and padding, but 
> that just takes up more memory).|||| Or is it MEDIA_BUS_FMT_SBGGR10_1X10 
> to describe the data on the bus correctly as 10bpp Bayer, and the odd 
> packing is ignored. Or do we need new enums?

Just add new enums to media-bus-format.h. It should be clear from comments and/or
the naming of the enum what the exact format it, so you'll need to think about
that carefully. Otherwise it's no big deal to add new formats there.

Regards,

	Hans

Re: Sony imx219 driver?

[Dave Stevenson]

Hi Hans.

On 22/07/16 10:46, Hans Verkuil wrote:
>
>
> On 07/21/2016 08:19 PM, Dave Stevenson wrote:
>> Hi All.
>>
>> Just a quick query to avoid duplicating effort. Has anyone worked on a
>> Sony IMX219 (or other Sony sensor) subdevice driver as yet?
>
> Not that I am aware of.

OK, glad to hear I won't be duplicating effort.

>> With the new Raspberry Pi camera being IMX219, and as Broadcom have
>> released an soc_camera based driver for the sensor already
>> (https://android.googlesource.com/kernel/bcm/+/android-bcm-tetra-3.10-lollipop-wear-release/drivers/media/video/imx219.c)
>
> #@!@$! Why do they never ever ask us what would be a good example driver to use. The
> soc-camera framework is deprecated and will likely be gone by the end of the year.

Yes, a touch grim.
I'm aware of the project it was done for from my time at Broadcom but 
wasn't directly involved. Partly in their defence:
(a) It was almost 3 years ago.
(b) It's on top of a 3.10 kernel, so lots of the nicer features that are 
now present just weren't there.
(c) However they did bastardise even soc_camera to extract extra info 
from the sensor driver for some of their upper layers. Not so nice.

>> I was going to investigate converting that to a subdevice. I just wanted
>> to check this wasn't already in someone else's work queue.
>>
>> A further Google shows that there's also an soc_camera IMX219 driver in
>> ChromiumOS, copyright Andrew Chew @ Nvidia, but author Guennadi
>> Liakhovetski who I know posts on here.
>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/factory-ryu-6486.14.B-chromeos-3.14/drivers/media/i2c/soc_camera/imx219.c.
>
> At least for the tegra there is a proper non-soc-camera driver in the works.

non-soc-camera driver for imx219, or for the tegra hardware blocks? I'm 
assuming the latter based on you saying you're not aware of an imx219 
driver being in progress.

As I have mentioned before, the other half of my playing is a subdevice 
driver for the CSI2 receiver on Pi. That bcm android kernel tree also 
released all the info needed for that block into the public domain, so 
I'm not even breaching NDAs in doing so :-) I hope to test my imx219 
driver against that.

>> The Broadcom one supports 8MPix and 1080P, the Chromium one only 8MP.
>> Perhaps a hybrid of the feature set?
>
> They usually only implement what they need, so I'd say for chromium they only needed 8MP support.
>
>> Throw in
>> https://github.com/ZenfoneArea/android_kernel_asus_zenfone5/blob/master/linux/modules/camera/drivers/media/i2c/imx219/imx219.h
>> as well, and we have register sets for numerous readout modes, plus
>> there are the ones in the Pi firmware which can be extracted if necessary.
>
> With all that code it shouldn't be hard to write a proper subdev driver.

Indeed.
I'll try to put something together and submit it to the list. I don't 
know what my timescales will end up being on it though.

>>
>> On a related note, if putting together a system with IMX219 or similar
>> producing Bayer raw 10, the data on the CSI2 bus is one of the
>> V4L2_PIX_FMT_SRGGB10P formats. What's the correct way to reflect that
>> from the sensor subdevice in an MEDIA_BUS_FMT_ enum?
>> The closest is MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE (or LE), but the data
>> isn't padded (the Pi CSI2 receiver can do the unpacking and padding, but
>> that just takes up more memory).|||| Or is it MEDIA_BUS_FMT_SBGGR10_1X10
>> to describe the data on the bus correctly as 10bpp Bayer, and the odd
>> packing is ignored. Or do we need new enums?
>
> Just add new enums to media-bus-format.h. It should be clear from comments and/or
> the naming of the enum what the exact format it, so you'll need to think about
> that carefully. Otherwise it's no big deal to add new formats there.

OK, will do.
I wasn't clear from reading the docs how accurately MEDIA_BUS_FMT_ was 
describing the pixel layout as it mentions NOT having a 1:1 mapping with 
V4L2_PIX_FMT_ enums. Having the clarification is great.

Thanks for the advice.
   Dave

Re: Sony imx219 driver?

[Sakari Ailus]

Hi Hans and Dave,

On Fri, Jul 22, 2016 at 11:46:59AM +0200, Hans Verkuil wrote:
> > On a related note, if putting together a system with IMX219 or similar 
> > producing Bayer raw 10, the data on the CSI2 bus is one of the 
> > V4L2_PIX_FMT_SRGGB10P formats. What's the correct way to reflect that 
> > from the sensor subdevice in an MEDIA_BUS_FMT_ enum?
> > The closest is MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE (or LE), but the data 
> > isn't padded (the Pi CSI2 receiver can do the unpacking and padding, but 
> > that just takes up more memory).|||| Or is it MEDIA_BUS_FMT_SBGGR10_1X10 
> > to describe the data on the bus correctly as 10bpp Bayer, and the odd 
> > packing is ignored. Or do we need new enums?
> 
> Just add new enums to media-bus-format.h. It should be clear from comments and/or
> the naming of the enum what the exact format it, so you'll need to think about
> that carefully. Otherwise it's no big deal to add new formats there.

The existing drivers that use 10-bit raw bayer formats on serial busses
(such as CSI-2) use 1X10 variants of the media bus formats. That's perhaps
not the neatest possible way to solve that, but that's what existing drivers
do. And doing differently breaks things.

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: Sony imx219 driver?

[Dave Stevenson]

Hi Ramiro

On 13/12/16 13:47, Ramiro Oliveira wrote:
> Hi Dave
>
> On 07/21/2016 08:19 PM, Dave Stevenson wrote:
>> Just a quick query to avoid duplicating effort. Has anyone worked on a
>> Sony IMX219 (or other Sony sensor) subdevice driver as yet?
>>
>> With the new Raspberry Pi camera being IMX219, and as Broadcom have
>> released an soc_camera based driver for the sensor already
>> (https://android.googlesource.com/kernel/bcm/+/android-bcm-tetra-3.10-lollipop-wear-release/drivers/media/video/imx219.c)
>> I was going to investigate converting that to a subdevice. I just wanted
>> to check this wasn't already in someone else's work queue.
>>
>> A further Google shows that there's also an soc_camera IMX219 driver in
>> ChromiumOS, copyright Andrew Chew @ Nvidia, but author Guennadi
>> Liakhovetski who I know posts on here.
>> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/factory-ryu-6486.14.B-chromeos-3.14/drivers/media/i2c/soc_camera/imx219.c.
>> The Broadcom one supports 8MPix and 1080P, the Chromium one only 8MP.
>> Perhaps a hybrid of the feature set? Throw in
>> https://github.com/ZenfoneArea/android_kernel_asus_zenfone5/blob/master/linux/modules/camera/drivers/media/i2c/imx219/imx219.h
>> as well, and we have register sets for numerous readout modes, plus
>> there are the ones in the Pi firmware which can be extracted if necessary.
>>
>> On a related note, if putting together a system with IMX219 or similar
>> producing Bayer raw 10, the data on the CSI2 bus is one of the
>> V4L2_PIX_FMT_SRGGB10P formats. What's the correct way to reflect that
>> from the sensor subdevice in an MEDIA_BUS_FMT_ enum?
>> The closest is MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE (or LE), but the data
>> isn't padded (the Pi CSI2 receiver can do the unpacking and padding, but
>> that just takes up more memory).|||| Or is it MEDIA_BUS_FMT_SBGGR10_1X10
>> to describe the data on the bus correctly as 10bpp Bayer, and the odd
>> packing is ignored. Or do we need new enums?
>
> Do you still plan on submitting this driver? If so, do you plan on submitting it
> soon?
>
> If you're not planning on submitting it any time soon, do you mind if submit it
> myself? I'm planning on having something ready for kernel submission by January.

I've got bogged down in other things and not got anywhere as yet, so go 
for it.
When I've got the CSI2 receiver subdevice working (hopefully soon) then 
I'll happily test your driver with it. For the time being I've been 
working against the TC358743 driver.

   Dave

> Thanks,
> Ramiro Oliveira
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: Sony imx219 driver?

[Ramiro Oliveira]

Hi Dave

On 07/21/2016 08:19 PM, Dave Stevenson wrote:
> Just a quick query to avoid duplicating effort. Has anyone worked on a
> Sony IMX219 (or other Sony sensor) subdevice driver as yet?
> 
> With the new Raspberry Pi camera being IMX219, and as Broadcom have
> released an soc_camera based driver for the sensor already
> (https://android.googlesource.com/kernel/bcm/+/android-bcm-tetra-3.10-lollipop-wear-release/drivers/media/video/imx219.c) 
> I was going to investigate converting that to a subdevice. I just wanted
> to check this wasn't already in someone else's work queue.
> 
> A further Google shows that there's also an soc_camera IMX219 driver in
> ChromiumOS, copyright Andrew Chew @ Nvidia, but author Guennadi
> Liakhovetski who I know posts on here.
> https://chromium.googlesource.com/chromiumos/third_party/kernel/+/factory-ryu-6486.14.B-chromeos-3.14/drivers/media/i2c/soc_camera/imx219.c. 
> The Broadcom one supports 8MPix and 1080P, the Chromium one only 8MP. 
> Perhaps a hybrid of the feature set? Throw in
> https://github.com/ZenfoneArea/android_kernel_asus_zenfone5/blob/master/linux/modules/camera/drivers/media/i2c/imx219/imx219.h 
> as well, and we have register sets for numerous readout modes, plus 
> there are the ones in the Pi firmware which can be extracted if necessary.
> 
> On a related note, if putting together a system with IMX219 or similar 
> producing Bayer raw 10, the data on the CSI2 bus is one of the 
> V4L2_PIX_FMT_SRGGB10P formats. What's the correct way to reflect that 
> from the sensor subdevice in an MEDIA_BUS_FMT_ enum?
> The closest is MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_BE (or LE), but the data 
> isn't padded (the Pi CSI2 receiver can do the unpacking and padding, but 
> that just takes up more memory).|||| Or is it MEDIA_BUS_FMT_SBGGR10_1X10 
> to describe the data on the bus correctly as 10bpp Bayer, and the odd 
> packing is ignored. Or do we need new enums?

Do you still plan on submitting this driver? If so, do you plan on submitting it
soon?

If you're not planning on submitting it any time soon, do you mind if submit it
myself? I'm planning on having something ready for kernel submission by January.

Thanks,
Ramiro Oliveira