Re: [PATCH v2 00/77] media: imx: Miscellaneous fixes and cleanups for i.MX7

From: Frieder Schrempf <frieder.schrempf@kontron.de>
To: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: linux-media@vger.kernel.org, Rui Miguel Silva <rmfrfs@gmail.com>,
	Steve Longerbeam <slongerbeam@gmail.com>,
	Philipp Zabel <p.zabel@pengutronix.de>,
	Ezequiel Garcia <ezequiel@collabora.com>,
	Fabio Estevam <festevam@gmail.com>
Subject: Re: [PATCH v2 00/77] media: imx: Miscellaneous fixes and cleanups for i.MX7
Date: Mon, 26 Apr 2021 10:06:51 +0200	[thread overview]
Message-ID: <5a600024-b5e9-e57f-f2c9-d75520666163@kontron.de> (raw)
In-Reply-To: <71888be2-faf3-f4c2-5cac-2a39acbeba81@kontron.de>

On 15.04.21 18:04, Frieder Schrempf wrote:
> On 15.04.21 11:00, Frieder Schrempf wrote:
>> Hi Laurent,
>>
>> On 19.03.21 02:27, Laurent Pinchart wrote:
>>> Hi Frieder,
>>>
>>> On Wed, Mar 17, 2021 at 07:04:19PM +0100, Frieder Schrempf wrote:
>>>> On 15.02.21 05:26, Laurent Pinchart wrote:
>>>>> Hello,
>>>>>
>>>>> This large patch series is a collection of miscellaneous fixes, 
>>>>> cleanups
>>>>> and enhancements for the i.MX7 camera support. Most notably, it
>>>>> implements support for the Media Controller API in the driver.
>>>>>
>>>>> Compared to v1, review comments have been taken into account, and the
>>>>> patches have been rebased on top of the DT bindings and latest imx
>>>>> changes as present in the linux-media tree. Patches 38/77, 39/77, 
>>>>> 60/77
>>>>> and 61/77 are new. For additional information, please see individual
>>>>> patches.
>>>>>
>>>>> I have successfully tested the code on an i.MX6ULL board (with an
>>>>> MT9M114 sensor), an I.MX7D board (with an IMX296 sensor), and an 
>>>>> i.MX8MM
>>>>> board (with an OV5640 sensor, and additional patches for i.MX8MM
>>>>> support).
>>>>
>>>> First of all, thanks for the great work!
>>>
>>> You're welcome.
>>>
>>>> I'm currently trying to get a setup with CSI + MIPI + ADV7280 
>>>> working on
>>>> an i.MX8MM system. Would you mind sharing the additional patches I need
>>>> for this?
>>>>
>>>> I guess you are referring to patches for the MIPI CSIS PHY and the
>>>> devicetree. Anything else, that I'm missing?
>>>
>>> I've pushed my work in progress patches to
>>>
>>>     git://linuxtv.org/pinchartl/media.git imx/next
>>>
>>> I haven't tested this on mainline though, as the i.MX8MM board I'm using
>>> currently requires an NXP BSP.
>>>
>>> For the DT integration, here's what I have in imx8mm.dtsi:
>>>
>> [...]
>>>
>>> Mainline seems to be missing the power domains, so you'll likely have to
>>> sort this out.
>>>
>>
>> Thanks a lot for the patches. I finally found some time to test this 
>> with our hardware. I'm working on v5.10 mainline and applied pending 
>> support for the power-domains from Lucas [1].
>>
>> It doesn't look bad and I can bring up the media devices, etc. but up 
>> to now I couldn't get any image from the ADV7280A-M.
>>
>> This is the first time I'm working with the media/v4l subsystem, so I 
>> might be missing some trivial things. Also I'm not quite sure if the 
>> adv7180 driver is currently fully compatible.
>>
>> Anyway here are some things I noticed. Maybe you could have a quick 
>> look and see if you can gather something from that, which helps me to 
>> get this running?
>>
>> After enabling the link to the adv7180 with:
>>
>> media-ctl -l "'adv7180 1-0021':0 -> 'imx7-mipi-csis.0':0[1]"
>>
>> all the links look ok, but not all the parameters seem to be 
>> propagated up to the subdevs (see [2]). But I don't even know if this 
>> should happen automatically.
>>
>> After setting up the format like this, the subdev setup looks ok as 
>> far as I can judge (see [3]):
>>
>> media-ctl -V "'adv7180 1-0021':0 [fmt:UYVY2X8/720x240 field:alternate]"
>> media-ctl -V "'imx7-mipi-csis.0':1 [fmt:UYVY2X8/720x240 field:alternate]"
>>
>> Also there is one small issue with the adv7180 not implementing 
>> neither V4L2_CID_LINK_FREQ nor V4L2_CID_PIXEL_RATE. So 
>> v4l2_get_link_rate() fails, but as this seems to be only used to 
>> calculate hs_settle, I just hardcoded the hs_settle value for now.
>>
>> Finally I don't get any image from the analog converter. The stream 
>> seems to be setup correctly, but I don't receive any frames. Though I 
>> can see that there is data on the MIPI CSI lanes.
> 
> I discovered, that when the stream is running and I toggle the 
> CSITX_PWRDN bit in the ADV7280 (disable and the enable again the MIPI 
> CSI TX), I receive a single frame from the camera, but nothing more 
> until I restart the streaming and toggle the CSITX_PWRDN again.
> 
> Also I'm not able to get any frame data if I try to get progressive 
> frames from the ADV (setting field = none and fmt = UYVY2X8/720x480), or 
> if I try to use PAL instead of NTSC.
> 
> So at least I know now that my hardware seems to work in general. But I 
> still hope to get this setup fully working...
> 

Just to wrap this up: It seems like for YUV422 you need some "special" 
settings.

First of all, just as with the RGB formats I needed to set 
MIPI_CSIS_ISPCFG_PIXEL_MODE_DUAL in the MIPI CSIS driver and 
BIT_TWO_8BIT_SENSOR in the CR3 register in the CSI driver.

Without this the MIPI CSIS hardware will always run into an overflow 
condition as soon as some data has been received.

Further I needed to set BIT_MIPI_DOUBLE_CMPNT in CR18 to get the colors 
right. No idea what it really does, but the RM mentions YUV422 in the 
description for this bit.

With this setup (and some changes on the adv7180 driver) I can get 
proper images from the camera. Though there's one problem remaining 
related to synchronization. It looks like the ADV outputs some invalid 
data at the beginning of the stream which results in an arbitrary 
horizontal and vertical offset of the image.

I guess that this could maybe be solved by implementing g_skip_frames() 
for the CSI driver, but I haven't tried that.

If anyone is interested, here is my WIP branch: 
https://git.kontron-electronics.de/linux/linux/-/commits/v5.10-mx8mm-csi.