From: Hans Verkuil <hverkuil@xs4all.nl>
To: Eugen.Hristev@microchip.com, jacopo@jmondi.org,
laurent.pinchart@ideasonboard.com
Cc: linux-media@vger.kernel.org, robh+dt@kernel.org,
sakari.ailus@iki.fi, linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org,
Nicolas.Ferre@microchip.com
Subject: Re: [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers
Date: Wed, 12 Jan 2022 10:52:16 +0100 [thread overview]
Message-ID: <aaa692f7-174a-83a9-8aae-64cda91044cb@xs4all.nl> (raw)
In-Reply-To: <5e4f591b-9105-9de0-5067-2bd82dc04983@microchip.com>
On 11/01/2022 17:38, Eugen.Hristev@microchip.com wrote:
> On 1/11/22 5:53 PM, Hans Verkuil wrote:
>> On 13/12/2021 14:49, Eugen Hristev wrote:
>>> During experiments with libcamera, it looks like vb2_is_streaming returns
>>> true before our start streaming is called.
>>> Order of operations is streamon -> queue -> start_streaming
>>> ISC would have started the DMA immediately when a buffer is being added
>>> to the vbqueue if the queue is streaming.
>>> It is more safe to start the DMA after the start streaming of the driver is
>>> called.
>>> Thus, even if vb2queue is streaming, add the buffer to the dma queue of the
>>> driver instead of actually starting the DMA process, if the start streaming
>>> has not been called yet.
>>>
>>> Signed-off-by: Eugen Hristev <eugen.hristev@microchip.com>
>>> ---
>>> drivers/media/platform/atmel/atmel-isc-base.c | 17 ++++++++++-------
>>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/media/platform/atmel/atmel-isc-base.c b/drivers/media/platform/atmel/atmel-isc-base.c
>>> index 26a6090f056c..e6c9071c04f0 100644
>>> --- a/drivers/media/platform/atmel/atmel-isc-base.c
>>> +++ b/drivers/media/platform/atmel/atmel-isc-base.c
>>> @@ -441,12 +441,14 @@ static void isc_buffer_queue(struct vb2_buffer *vb)
>>> unsigned long flags;
>>>
>>> spin_lock_irqsave(&isc->dma_queue_lock, flags);
>>> - if (!isc->cur_frm && list_empty(&isc->dma_queue) &&
>>> - vb2_is_streaming(vb->vb2_queue)) {
>>> +
>>> + if (!isc->cur_frm && list_empty(&isc->dma_queue) && !isc->stop) {
>>> isc->cur_frm = buf;
>>> isc_start_dma(isc);
>>> - } else
>>> + } else {
>>> list_add_tail(&buf->list, &isc->dma_queue);
>>> + }
>>> +
>>> spin_unlock_irqrestore(&isc->dma_queue_lock, flags);
>>> }
>>
>> Both the old and new code doesn't make a lot of sense.
>
> Hello Hans,
>
> I debated this a bit with Laurent when I initially wrote the patch.
>
> The problem with the current code, or the current way things are
> working, is the fact that vb2_is_streaming returns true from the moment
> of streamon.
> The DMA engine of the ISC is being started at the moment of
> start_streaming, and if buffers are being sent earlier/later, they are
> just added to the queue and DMA engine is restarted if it finished
> previously.
> However if frames are added beforehand, like, first streamon, then add
beforehand -> afterwards?
> some frames, and then later call the start_streaming, ISC will crash.
> (as I said, DMA engine is prepared at start_streaming).
> To avoid this, I changed the vb2_is_streaming calls to keeping an
> internal state of the streaming.
> Do you think I should try another different approach to solve this ?
> Perhaps do not prepare the ISC dma engine at start_streaming, but at
> another place ?
>
> Laurent do you remember the talk we had about this problem a while back ?
>
>>
>> buf_queue is only called by vb2 if start_streaming has already been called or is
>> about to be called.
>
> That's a huge difference for ISC driver as it is today. start_streaming
> being called afterwards, means the ISC DMA engine is not started (not
> started streaming), hence a crash.
The start_streaming op is called when streamon is called and if there are
sufficient buffers (>= q->min_buffers_needed). If there are insufficient
buffers at streamon time, then start_streaming is only called when the number
of queued buffers reaches the minimum.
vb2 has three different state checks that you can use:
vb2_is_busy() is true if buffers are allocated. Typically this means that you
can no longer do format changes or anything that might change the buffer
sizes or layout.
vb2_is_streaming() is true if streamon was called. If this is true, then vb2_is_busy()
is also true.
And finally vb2_start_streaming_called() is true if the start_streaming op
was called (i.e. DMA is in progress). If this is true, then vb2_is_streaming is
also true.
Basically vb2_is_streaming() reports the streaming state that userspace sees,
while vb2_start_streaming_called() reports the internal DMA streaming state.
The confusion is probably that you were using vb2_is_streaming instead of
vb2_start_streaming_called.
>
>>
>> Typically all that the buf_queue op does is to call list_add_tail(&buf->list, &isc->dma_queue);
>> inside the spinlock.
>>
>>>
>>> @@ -1014,7 +1016,7 @@ static int isc_s_fmt_vid_cap(struct file *file, void *priv,
>>> {
>>> struct isc_device *isc = video_drvdata(file);
>>>
>>> - if (vb2_is_streaming(&isc->vb2_vidq))
>>> + if (!isc->stop)
>>
>> This is weird as well. Normally this calls vb2_is_busy to check if the
>> queue is busy (that really means that buffers are already allocated, so
>> changing the format isn't allowed anymore).
>
> You think the query to the streaming status makes no sense here hence it
> should be removed completely ?
Right. And instead check vb2_is_busy().
I actually think that this is something that would fail with v4l2-compliance.
Do you still run v4l2-compliance?
> I can do that with a prequel patch if it's the case.
Probably wise.
Regards,
Hans
>
> Thanks for reviewing,
>
> Eugen
>
>>
>>> return -EBUSY;
>>>
>>> return isc_set_fmt(isc, f);
>>> @@ -1536,7 +1538,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>>
>>> isc_update_awb_ctrls(isc);
>>>
>>> - if (vb2_is_streaming(&isc->vb2_vidq)) {
>>> + if (!isc->stop) {
>>
>> Ditto.
>>
>>> /*
>>> * If we are streaming, we can update profile to
>>> * have the new settings in place.
>>> @@ -1552,8 +1554,7 @@ static int isc_s_awb_ctrl(struct v4l2_ctrl *ctrl)
>>> }
>>>
>>> /* if we have autowhitebalance on, start histogram procedure */
>>> - if (ctrls->awb == ISC_WB_AUTO &&
>>> - vb2_is_streaming(&isc->vb2_vidq) &&
>>> + if (ctrls->awb == ISC_WB_AUTO && !isc->stop &&
>>> ISC_IS_FORMAT_RAW(isc->config.sd_format->mbus_code))
>>> isc_set_histogram(isc, true);
>>>
>>> @@ -1829,6 +1830,8 @@ static int isc_async_complete(struct v4l2_async_notifier *notifier)
>>> struct vb2_queue *q = &isc->vb2_vidq;
>>> int ret = 0;
>>>
>>> + isc->stop = true;
>>> +
>>
>> I'm really not sure that you need the stop bool at all.
>>
>>> INIT_WORK(&isc->awb_work, isc_awb_work);
>>>
>>> ret = v4l2_device_register_subdev_nodes(&isc->v4l2_dev);
>>
>> Regards,
>>
>> Hans
>>
>
next prev parent reply other threads:[~2022-01-12 9:52 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-13 13:49 [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 01/23] MAINTAINERS: add microchip csi2dc Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 02/23] dt-bindings: media: atmel: csi2dc: add bindings for " Eugen Hristev
2021-12-15 20:16 ` Rob Herring
2021-12-13 13:49 ` [PATCH v3 03/23] media: atmel: introduce microchip csi2dc driver Eugen Hristev
2022-01-11 15:43 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 04/23] media: atmel: atmel-isc: split the clock code into separate source file Eugen Hristev
2022-01-11 15:43 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 05/23] media: atmel: atmel-isc: replace video device name with module name Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 06/23] media: atmel: atmel-sama7g5-isc: fix ispck leftover Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 07/23] media: atmel: atmel-isc-base: use streaming status when queueing buffers Eugen Hristev
2022-01-11 15:53 ` Hans Verkuil
2022-01-11 16:38 ` Eugen.Hristev
2022-01-12 9:52 ` Hans Verkuil [this message]
2021-12-13 13:49 ` [PATCH v3 08/23] media: atmel: atmel-isc-base: remove frameintervals VIDIOC Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 09/23] media: atmel: atmel-isc-base: report frame sizes as full supported range Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 10/23] media: atmel: atmel-isc-base: implement mbus_code support in enumfmt Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 11/23] media: atmel: atmel-isc-base: fix bytesperline value for planar formats Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 12/23] media: atmel: atmel-isc: implement media controller Eugen Hristev
2022-01-11 16:06 ` Jacopo Mondi
2022-01-11 16:30 ` Eugen.Hristev
2022-01-11 16:51 ` Jacopo Mondi
2022-01-11 17:00 ` Eugen.Hristev
2021-12-13 13:49 ` [PATCH v3 13/23] ARM: dts: at91: sama7g5: add nodes for video capture Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 14/23] ARM: configs: at91: sama7: add xisc and csi2dc Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 15/23] ARM: multi_v7_defconfig: add atmel video pipeline modules Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 16/23] media: atmel: atmel-sama5d2-isc: fix wrong mask in YUYV format check Eugen Hristev
2021-12-13 13:49 ` [PATCH v3 17/23] media: atmel: atmel-isc-base: use mutex to lock awb workqueue from streaming Eugen Hristev
2022-01-12 8:58 ` Jacopo Mondi
2022-01-19 15:40 ` Eugen.Hristev
2021-12-13 13:49 ` [PATCH v3 18/23] media: atmel: atmel-isc-base: add wb debug messages Eugen Hristev
2022-01-12 9:00 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 19/23] media: atmel: atmel-isc-base: clamp wb gain coefficients Eugen Hristev
2022-01-12 9:04 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 20/23] media: atmel: atmel-sama7g5-isc: fix UYVY input format mbus_code typo Eugen Hristev
2022-01-12 9:04 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 21/23] media: atmel: atmel-isc: add raw Bayer 8bit 10bit output formats Eugen Hristev
2022-01-12 9:05 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 22/23] media: atmel: atmel-isc: compact the controller formats list Eugen Hristev
2022-01-12 9:06 ` Jacopo Mondi
2021-12-13 13:49 ` [PATCH v3 23/23] media: atmel: atmel-isc: change format propagation to subdev into only verification Eugen Hristev
2022-01-12 9:21 ` Jacopo Mondi
2022-01-20 8:43 ` Eugen.Hristev
2022-01-20 8:48 ` Eugen.Hristev
2022-01-20 8:58 ` Jacopo Mondi
2022-01-12 12:07 ` [PATCH v3 00/23] media: atmel: atmel-isc: implement media controller Hans Verkuil
2022-01-12 12:46 ` Eugen.Hristev
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aaa692f7-174a-83a9-8aae-64cda91044cb@xs4all.nl \
--to=hverkuil@xs4all.nl \
--cc=Eugen.Hristev@microchip.com \
--cc=Nicolas.Ferre@microchip.com \
--cc=jacopo@jmondi.org \
--cc=laurent.pinchart@ideasonboard.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=robh+dt@kernel.org \
--cc=sakari.ailus@iki.fi \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).