From: Nicolas Dufresne <nicolas.dufresne@collabora.com> To: Alex Bee <knaerzche@gmail.com>, linux-media@vger.kernel.org, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>, Mauro Carvalho Chehab <mchehab@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: kernel@collabora.com, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v1 4/5] media: rkvdec: Re-enable H.264 error detection Date: Mon, 13 Jun 2022 09:09:56 -0400 [thread overview] Message-ID: <c9ecb36f3490326f67ff84515a8aee4d264e4361.camel@collabora.com> (raw) In-Reply-To: <8efa6811-ee17-4dd2-23a7-e0471af8c0a6@gmail.com> Le samedi 11 juin 2022 à 14:08 +0200, Alex Bee a écrit : > Am 10.06.22 um 14:52 schrieb Nicolas Dufresne: > > This re-enables H.264 error detection, but using the other error mode. > > In that mode, the decoder will skip over the error macro-block or > > slices and complete the decoding. As a side effect, the error status > > is not set in the interrupt status register, and instead errors are > > detected per format. Using this mode workaround the issue that the > > HW get stuck in error stated and allow reporting that some corruption > > may be present in the buffer returned to userland. > > > > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> > > --- > > drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++++++++--- > > 1 file changed, 20 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c > > index 55596ce6bb6e..60a89918e2c1 100644 > > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c > > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c > > @@ -1175,14 +1175,15 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) > > > > schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000)); > > > > - writel(0, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN); > > - writel(0, rkvdec->regs + RKVDEC_REG_H264_ERR_E); > > + writel(0xffffffff, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN); > > + writel(0xffffffff, rkvdec->regs + RKVDEC_REG_H264_ERR_E); > > writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND); > > writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND); > > > > /* Start decoding! */ > > writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E | > > - RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E, > > + RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E | > > + RKVDEC_H264ORVP9_ERR_MODE, > > rkvdec->regs + RKVDEC_REG_INTERRUPT); > > > > return 0; > > @@ -1196,10 +1197,26 @@ static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) > > return 0; > > } > > > > +static int rkvdec_h264_check_error_info(struct rkvdec_ctx *ctx) > > +{ > > + struct rkvdec_dev *rkvdec = ctx->dev; > > + int err; > > + > > + err = readl(rkvdec->regs + RKVDEC_REG_H264_ERRINFO_NUM); > > + if (err & RKVDEC_STRMD_DECT_ERR_FLAG) { > > + pr_debug("Decoded picture have %i/%i slices with errors.\n", > > + RKVDEC_ERR_PKT_NUM(err), RKVDEC_SLICEDEC_NUM(err)); > > + return VB2_BUF_STATE_ERROR; > > + } > > + > > + return VB2_BUF_STATE_DONE; > > +} > > + > > const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = { > > .adjust_fmt = rkvdec_h264_adjust_fmt, > > .start = rkvdec_h264_start, > > .stop = rkvdec_h264_stop, > > .run = rkvdec_h264_run, > > .try_ctrl = rkvdec_h264_try_ctrl, > > + .check_error_info = rkvdec_h264_check_error_info, > > }; > > Actually I'm not sure I fully understand what you are expecting the > userspace to do with the information that there was an (HW!) error, > which might or might not be bitstrean related. Resending the > corrupted(?) frame until the HW fully hangs? > As the interrupt reports an HW error it should (at least also) be > handled driver-side and the HW is known not to be able to fully reset > itself in case of an error. > I think this will make behavior worse than it is now (for real-life > users) where errors are eventually just ignored. I've changed the decoding mode, see bit 19 or swreg1. In that mode, the decoder will behave just like if error detection was off. It will just keep going and produce "something". With the set of corrupted streams we had, we found that the decoder no longer get stuck, and we are aware of the possibly corrupted buffers. In current mode, the decoder tries to stop as soon as an error is met, which most of the time means nothing is every written to the buffer. And as you mention, it often fail at "self reset". In streaming there is two style of handling corrupted buffers, one is to skip until valid state, and the other is to show them even if corrupted. In stack like GStreamer we just flag the corrupted frames based on the ERROR flag (unless payload size is 0) and let the user chose to drop them or not. > I think this will make behavior worse than it is now (for real-life users) where errors are eventually just ignored. Just ignoring the errors is way better then an infinite row of errors. At the moment, FFMPEG/Chromium and GStreamer ignores errors indeed. I got some work in progress patch in GStreamer I've used to test this, but its not ready yet. In the current behaviour, if you hit an error, you basically have 9 chances out of 10 to keep replaying ancient buffers in loop till the end of time. This is because the self reset never completes, and you get the same error over and over regardless what you pass to the decoder. regards, Nicolas p.s. the tests should land (if not already) in ChromeOS taste suite. > > Alex
WARNING: multiple messages have this Message-ID (diff)
From: Nicolas Dufresne <nicolas.dufresne@collabora.com> To: Alex Bee <knaerzche@gmail.com>, linux-media@vger.kernel.org, Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>, Mauro Carvalho Chehab <mchehab@kernel.org>, Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: kernel@collabora.com, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v1 4/5] media: rkvdec: Re-enable H.264 error detection Date: Mon, 13 Jun 2022 09:09:56 -0400 [thread overview] Message-ID: <c9ecb36f3490326f67ff84515a8aee4d264e4361.camel@collabora.com> (raw) In-Reply-To: <8efa6811-ee17-4dd2-23a7-e0471af8c0a6@gmail.com> Le samedi 11 juin 2022 à 14:08 +0200, Alex Bee a écrit : > Am 10.06.22 um 14:52 schrieb Nicolas Dufresne: > > This re-enables H.264 error detection, but using the other error mode. > > In that mode, the decoder will skip over the error macro-block or > > slices and complete the decoding. As a side effect, the error status > > is not set in the interrupt status register, and instead errors are > > detected per format. Using this mode workaround the issue that the > > HW get stuck in error stated and allow reporting that some corruption > > may be present in the buffer returned to userland. > > > > Signed-off-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> > > --- > > drivers/staging/media/rkvdec/rkvdec-h264.c | 23 +++++++++++++++++++--- > > 1 file changed, 20 insertions(+), 3 deletions(-) > > > > diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c > > index 55596ce6bb6e..60a89918e2c1 100644 > > --- a/drivers/staging/media/rkvdec/rkvdec-h264.c > > +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c > > @@ -1175,14 +1175,15 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) > > > > schedule_delayed_work(&rkvdec->watchdog_work, msecs_to_jiffies(2000)); > > > > - writel(0, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN); > > - writel(0, rkvdec->regs + RKVDEC_REG_H264_ERR_E); > > + writel(0xffffffff, rkvdec->regs + RKVDEC_REG_STRMD_ERR_EN); > > + writel(0xffffffff, rkvdec->regs + RKVDEC_REG_H264_ERR_E); > > writel(1, rkvdec->regs + RKVDEC_REG_PREF_LUMA_CACHE_COMMAND); > > writel(1, rkvdec->regs + RKVDEC_REG_PREF_CHR_CACHE_COMMAND); > > > > /* Start decoding! */ > > writel(RKVDEC_INTERRUPT_DEC_E | RKVDEC_CONFIG_DEC_CLK_GATE_E | > > - RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E, > > + RKVDEC_TIMEOUT_E | RKVDEC_BUF_EMPTY_E | > > + RKVDEC_H264ORVP9_ERR_MODE, > > rkvdec->regs + RKVDEC_REG_INTERRUPT); > > > > return 0; > > @@ -1196,10 +1197,26 @@ static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) > > return 0; > > } > > > > +static int rkvdec_h264_check_error_info(struct rkvdec_ctx *ctx) > > +{ > > + struct rkvdec_dev *rkvdec = ctx->dev; > > + int err; > > + > > + err = readl(rkvdec->regs + RKVDEC_REG_H264_ERRINFO_NUM); > > + if (err & RKVDEC_STRMD_DECT_ERR_FLAG) { > > + pr_debug("Decoded picture have %i/%i slices with errors.\n", > > + RKVDEC_ERR_PKT_NUM(err), RKVDEC_SLICEDEC_NUM(err)); > > + return VB2_BUF_STATE_ERROR; > > + } > > + > > + return VB2_BUF_STATE_DONE; > > +} > > + > > const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = { > > .adjust_fmt = rkvdec_h264_adjust_fmt, > > .start = rkvdec_h264_start, > > .stop = rkvdec_h264_stop, > > .run = rkvdec_h264_run, > > .try_ctrl = rkvdec_h264_try_ctrl, > > + .check_error_info = rkvdec_h264_check_error_info, > > }; > > Actually I'm not sure I fully understand what you are expecting the > userspace to do with the information that there was an (HW!) error, > which might or might not be bitstrean related. Resending the > corrupted(?) frame until the HW fully hangs? > As the interrupt reports an HW error it should (at least also) be > handled driver-side and the HW is known not to be able to fully reset > itself in case of an error. > I think this will make behavior worse than it is now (for real-life > users) where errors are eventually just ignored. I've changed the decoding mode, see bit 19 or swreg1. In that mode, the decoder will behave just like if error detection was off. It will just keep going and produce "something". With the set of corrupted streams we had, we found that the decoder no longer get stuck, and we are aware of the possibly corrupted buffers. In current mode, the decoder tries to stop as soon as an error is met, which most of the time means nothing is every written to the buffer. And as you mention, it often fail at "self reset". In streaming there is two style of handling corrupted buffers, one is to skip until valid state, and the other is to show them even if corrupted. In stack like GStreamer we just flag the corrupted frames based on the ERROR flag (unless payload size is 0) and let the user chose to drop them or not. > I think this will make behavior worse than it is now (for real-life users) where errors are eventually just ignored. Just ignoring the errors is way better then an infinite row of errors. At the moment, FFMPEG/Chromium and GStreamer ignores errors indeed. I got some work in progress patch in GStreamer I've used to test this, but its not ready yet. In the current behaviour, if you hit an error, you basically have 9 chances out of 10 to keep replaying ancient buffers in loop till the end of time. This is because the self reset never completes, and you get the same error over and over regardless what you pass to the decoder. regards, Nicolas p.s. the tests should land (if not already) in ChromeOS taste suite. > > Alex _______________________________________________ Linux-rockchip mailing list Linux-rockchip@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-rockchip
next prev parent reply other threads:[~2022-06-13 13:10 UTC|newest] Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-06-10 12:52 [PATCH v1 0/5] media: rkvdec: Fix H.264 error resilience Nicolas Dufresne 2022-06-10 12:52 ` [PATCH v1 1/5] media: rkvdec: Disable H.264 error detection Nicolas Dufresne 2022-06-10 12:52 ` Nicolas Dufresne 2022-06-10 13:26 ` Dmitry Osipenko 2022-06-10 13:26 ` Dmitry Osipenko 2022-06-10 16:39 ` Brian Norris 2022-06-10 16:39 ` Brian Norris 2022-06-27 17:44 ` Ezequiel Garcia 2022-06-27 17:44 ` Ezequiel Garcia 2022-06-10 12:52 ` [PATCH v1 2/5] media: rkvdec: Add an ops to check for decode errors Nicolas Dufresne 2022-06-10 12:52 ` Nicolas Dufresne 2022-06-14 14:44 ` Hans Verkuil 2022-06-14 14:44 ` Hans Verkuil 2022-06-14 16:14 ` Nicolas Dufresne 2022-06-14 16:14 ` Nicolas Dufresne 2022-11-24 10:28 ` Hans Verkuil 2022-11-24 10:28 ` Hans Verkuil 2022-06-10 12:52 ` [PATCH v1 3/5] media: rkvdec: Fix RKVDEC_ERR_PKT_NUM macro Nicolas Dufresne 2022-06-10 12:52 ` Nicolas Dufresne 2022-06-10 12:52 ` [PATCH v1 4/5] media: rkvdec: Re-enable H.264 error detection Nicolas Dufresne 2022-06-10 12:52 ` Nicolas Dufresne 2022-06-10 13:20 ` Dan Carpenter 2022-06-10 13:20 ` Dan Carpenter 2022-06-10 13:48 ` Dmitry Osipenko 2022-06-10 13:48 ` Dmitry Osipenko 2022-06-10 16:23 ` Nicolas Dufresne 2022-06-10 16:23 ` Nicolas Dufresne 2022-06-10 15:01 ` Ezequiel Garcia 2022-06-10 15:01 ` Ezequiel Garcia 2022-06-10 16:38 ` Nicolas Dufresne 2022-06-10 16:38 ` Nicolas Dufresne 2022-06-11 12:08 ` Alex Bee 2022-06-11 12:08 ` Alex Bee 2022-06-13 13:09 ` Nicolas Dufresne [this message] 2022-06-13 13:09 ` Nicolas Dufresne 2022-06-10 12:52 ` [PATCH v1 5/5] media: rkvdec: Improve error handling Nicolas Dufresne 2022-06-10 12:52 ` Nicolas Dufresne 2022-06-10 19:14 ` Sebastian Fricke 2022-06-10 19:14 ` Sebastian Fricke
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=c9ecb36f3490326f67ff84515a8aee4d264e4361.camel@collabora.com \ --to=nicolas.dufresne@collabora.com \ --cc=ezequiel@vanguardiasur.com.ar \ --cc=gregkh@linuxfoundation.org \ --cc=kernel@collabora.com \ --cc=knaerzche@gmail.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-media@vger.kernel.org \ --cc=linux-rockchip@lists.infradead.org \ --cc=linux-staging@lists.linux.dev \ --cc=mchehab@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.