From: Paul Menzel <pmenzel@molgen.mpg.de>
To: Frederic Chen <frederic.chen@mediatek.com>
Cc: hans.verkuil@cisco.com,
laurent.pinchart+renesas@ideasonboard.com, tfiga@chromium.org,
matthias.bgg@gmail.com, mchehab@kernel.org, yuzhao@chromium.org,
zwisler@chromium.org, linux-mediatek@lists.infradead.org,
linux-arm-kernel@lists.infradead.org, Sean.Cheng@mediatek.com,
sj.huang@mediatek.com, christie.yu@mediatek.com,
holmes.chiou@mediatek.com, Jerry-ch.Chen@mediatek.com,
jungo.lin@mediatek.com, Rynn.Wu@mediatek.com,
linux-media@vger.kernel.org, srv_heupstream@mediatek.com,
devicetree@vger.kernel.org, shik@chromium.org,
suleiman@chromium.org, Allan.Yang@mediatek.com,
Sakari Ailus <sakari.ailus@iki.fi>,
Guenter Roeck <groeck@chromium.org>
Subject: Re: [RFC PATCH V3 0/5] media: platform: Add support for Digital Image Processing (DIP) on mt8183 SoC
Date: Sat, 19 Aug 2023 07:43:37 +0200 [thread overview]
Message-ID: <7f8cb8c7-cbf2-40c9-ac8f-c3c9b97919d7@molgen.mpg.de> (raw)
In-Reply-To: <20190909192244.9367-1-frederic.chen@mediatek.com>
[Cc: +Sakari, +Guenter]
Dear Frederic, dear Linux folks,
Am 09.09.19 um 21:22 schrieb frederic.chen@mediatek.com:
> This RFC patch series added Digital Image Processing (DIP) driver on Mediatek
> mt8183 SoC. It belongs to the Mediatek's ISP driver series based on V4L2 and
> media controller framework. I posted the main part of the DIP driver as RFC to
> discuss first and would like some review comments.
>
> I appreciate the comment of Tomasz in RFC V2. The RFC V3 patch addressed on all
> issues reviewed in V2 except the one about Mediatek proprietary MDP stride,
> depth and raw depth usage which is still under discussion. I will refactor
> the related parts once we come to the conclusion.
>
> You can check the following URL for the detail.
> http://lists.infradead.org/pipermail/linux-mediatek/2019-September/023254.html
>
>
> In V3, I also removed all workaround solution about the following V4L2
> compliance tool issues so that we got the related failed result.
>
> 1. Request API test doesn't know which buffers of the video devices are
> required so we got failed in testRequests()
>
> 2. V4L2 compliance test check if the driver return error when passing an
> invalid image size, but in vb2_create_bufs() case, we don't know if the
> size check is required or not.
>
> Please see the following URL for the detail.
> http://lists.infradead.org/pipermail/linux-mediatek/2019-June/020884.html
>
>
> Besides that, we got a new issue about the test case. When receiving the
> VIDIOC_SUBDEV_G_FMT ioctl on a DIP sub device's pad which connects with a
> meta video device, we return -EINVEL since it doesn't represent an image
> data flow (no width and height information), but the test case expects
> that the driver return some media format information.
>
> Sub-Device ioctls (Sink Pad 1):
> fail: v4l2-test-subdevs.cpp(352): doioctl(node, VIDIOC_SUBDEV_G_FMT, &fmt)
> test Try VIDIOC_SUBDEV_G/S_FMT: FAIL
>
>
> ==============
> Introduction
> ==============
>
> Digital Image Processing (DIP) unit can accept the tuning parameters and
> adjust the image content in Mediatek ISP system. Furthermore, it performs
> demosaicing and noise reduction on the image to support the advanced camera
> features of the application. The DIP driver also support image format
> conversion, resizing and rotation with its hardware path.
>
> The driver is implemented with V4L2 and media controller framework. We
> have the following entities describing the DIP path. Since a DIP frame has
> multiple buffers, the driver uses Request API to control the multiple
> buffer's enqueue flow.
>
> 1. Meta (output video device): connects to DIP sub device. It accepts the
> input tuning buffer from userspace. The metadata interface used currently
> is only a temporary solution to kick off driver development and is not
> ready for reviewed yet.
>
> 2. RAW (output video device): connects to DIP sub device. It accepts input
> image buffer from userspace.
>
> 3. DIP (sub device): connects to MDP-0 and MDP-1. When processing an image,
> DIP hardware support multiple output images with different size and format
> so it needs two capture video devices to return the streaming data to the
> user.
>
> 4. MDP-0 (capture video device): return the processed image data.
>
> 5. MDP-1 (capture video device): return the processed image data, the
> image size and format can be different from the ones of MDP-0.
>
> The overall file structure of the DIP driver is as following:
>
> * mtk_dip-v4l2.c: implements DIP platform driver, V4L2 and vb2 operations.
>
> * mtk_dip-sys.c: implements the hardware job handling flow including the part of
> interaction with the SCP and MDP.
>
> * mtk_dip-dev.c: implements dip pipe utilities. DIP driver supports 3 software
> pipes (preview, capture and reprocessing) at the same time. All
> the pipes share the same DIP hardware to process the images.
Thank you for your work. I use the Lenovo IdeaPad Duet Chromebook
(google/kukui variant of google/krane), and noticed the messages below
using the camera in the browser with recent ChromeOS:
[ 0.000000] Linux version 5.10.180-22631-gc8e37fc5f0ab
(chrome-bot@chromeos-release-builder-us-central1-b-x32-66-okmh)
(Chromium OS 17.0_pre496208_p20230501-r6 clang version 17.0.0
(/mnt/host/source/src/third_party/llvm-project
98f5a340975bc00197c57e39eb4ca26e2da0e8a2), LLD 17.0.0) #1 SMP PREEMPT
Wed Jul 26 19:01:55 PDT 2023
[…]
[ 2766.733517] mtk-cam-dip 15022000.dip: req(0xffffff8e5fdc9800),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.737034] req(0xffffff8e5fdc9800),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.772352] mtk-cam-dip 15022000.dip: req(0xffffff8d88002000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.775906] req(0xffffff8d88002000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.788790] mtk-cam-dip 15022000.dip: req(0xffffff8d88000000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.792327] req(0xffffff8d88000000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.830257] mtk-cam-dip 15022000.dip: req(0xffffff8e5ff46000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.833806] req(0xffffff8e5ff46000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.869589] mtk-cam-dip 15022000.dip: req(0xffffff8e5ff44000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.873104] req(0xffffff8e5ff44000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.889804] mtk-cam-dip 15022000.dip: req(0xffffff8e5ff41000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.893351] req(0xffffff8e5ff41000),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.939293] mtk-cam-dip 15022000.dip: req(0xffffff8e5ff43800),
req->dip_pipe(0xffffff8d829a0398)
[ 2766.942827] req(0xffffff8e5ff43800),
req->dip_pipe(0xffffff8d829a0398)
Search for that in the upstream Linux kernel, I found out, the the DIP
support is not upstreamed yet, and also has not been added to the
Chromium OS Linux kernel branches chromeos-5.15 and chromeos-6.1 [1].
Were you able to come to a conclusion regarding the two(?) issues
mentioned in your cover letter, so this series can be re-posted as
non-RFC? The driver seems to work well on millions(?) of devices, so
it’d be great to have it upstream.
[…]
Kind regards,
Paul
[1]:
https://chromium-review.googlesource.com/q/I1d1ba58cbdcdcc161b140398fc26b24ec2134cdb
prev parent reply other threads:[~2023-08-19 5:45 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-09 19:22 [RFC PATCH V3 0/5] media: platform: Add support for Digital Image Processing (DIP) on mt8183 SoC frederic.chen
2019-09-09 19:22 ` [RFC PATCH V3 1/5] dt-bindings: mt8183: Added DIP dt-bindings frederic.chen
2019-09-13 21:48 ` Rob Herring
2019-10-02 9:22 ` Sakari Ailus
2019-09-09 19:22 ` [RFC PATCH V3 2/5] dts: arm64: mt8183: Add DIP nodes frederic.chen
2019-09-09 19:22 ` [RFC PATCH V3 3/5] media: platform: Add Mediatek DIP driver KConfig frederic.chen
2019-10-02 9:20 ` Sakari Ailus
2019-09-09 19:22 ` [RFC PATCH V3 4/5] platform: mtk-isp: Add Mediatek DIP driver frederic.chen
2019-09-10 4:04 ` Tomasz Figa
2019-09-11 17:41 ` Frederic Chen
2019-09-12 5:58 ` Tomasz Figa
2019-09-19 9:41 ` Frederic Chen
2019-09-19 9:44 ` Tomasz Figa
2019-09-23 12:23 ` Sakari Ailus
2019-11-14 4:58 ` Pi-Hsun Shih
2019-09-09 19:22 ` [RFC PATCH V3 5/5] media: platform: mtk-mdp3: Add struct tuning_addr and img_sw_buffer frederic.chen
2019-10-02 9:21 ` Sakari Ailus
2019-09-23 12:11 ` [RFC PATCH V3 0/5] media: platform: Add support for Digital Image Processing (DIP) on mt8183 SoC Sakari Ailus
2023-08-19 5:43 ` Paul Menzel [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7f8cb8c7-cbf2-40c9-ac8f-c3c9b97919d7@molgen.mpg.de \
--to=pmenzel@molgen.mpg.de \
--cc=Allan.Yang@mediatek.com \
--cc=Jerry-ch.Chen@mediatek.com \
--cc=Rynn.Wu@mediatek.com \
--cc=Sean.Cheng@mediatek.com \
--cc=christie.yu@mediatek.com \
--cc=devicetree@vger.kernel.org \
--cc=frederic.chen@mediatek.com \
--cc=groeck@chromium.org \
--cc=hans.verkuil@cisco.com \
--cc=holmes.chiou@mediatek.com \
--cc=jungo.lin@mediatek.com \
--cc=laurent.pinchart+renesas@ideasonboard.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-mediatek@lists.infradead.org \
--cc=matthias.bgg@gmail.com \
--cc=mchehab@kernel.org \
--cc=sakari.ailus@iki.fi \
--cc=shik@chromium.org \
--cc=sj.huang@mediatek.com \
--cc=srv_heupstream@mediatek.com \
--cc=suleiman@chromium.org \
--cc=tfiga@chromium.org \
--cc=yuzhao@chromium.org \
--cc=zwisler@chromium.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).