* [RFC PATCH 0/5] Rockchip ISP1 Driver
@ 2017-11-15 7:29 Jacob Chen
2017-11-15 7:29 ` [RFC PATCH 1/5] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format Jacob Chen
2017-11-15 8:03 ` [RFC PATCH 0/5] Rockchip ISP1 Driver Tomasz Figa
0 siblings, 2 replies; 4+ messages in thread
From: Jacob Chen @ 2017-11-15 7:29 UTC (permalink / raw)
To: linux-rockchip
Cc: linux-media, linux-kernel, linux-arm-kernel, heiko, mchehab,
laurent.pinchart+renesas, hans.verkuil, tfiga, nicolas,
sakari.ailus, zhengsq, zyc, eddie.cai.linux, jeffy.chen,
allon.huang, p.zabel, slongerbeam, linux, Jacob Chen
This patch series add a ISP(Camera) v4l2 driver for rockchip rk3288/rk3399 SoC.
TODO:
- Thomas is rewriting the binding code between isp, phy, sensors, i hope we could get suggestions.
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/768633/2
rules:
- There are many mipi interfaces("rx0", "dxrx0")(actually it also could be parallel interface) in SoC and isp can decide which one will be used.
- Sometimes there will be more than one senor in a mipi phy, the sofrware should decide which one is used(media link).
- rk3399 have two isp.
- Add a dummy buffer(dma_alloc_coherent) so drvier won't hold buffer.
- Finish all TODO comments(mostly about hardware) in driver.
To help do a quick review, i have push source code to my Github.
https://github.com/wzyy2/linux/tree/rkisp1/drivers/media/platform/rockchip/isp1
Below are some infomations about driver/hardware:
Rockchip ISP1 have many Hardware Blocks(simplied):
MIPI --> ISP --> DCrop(Mainpath) --> RSZ(Mainpath) --> DMA(Mainpath)
DMA-Input --> --> DCrop(Selfpath) --> RSZ(Selfpath) --> DMA(Selfpath);)
(Acutally the TRM(rk3288, isp) could be found online...... which contains a more detailed block diagrams ;-P)
The funcitons of each hardware block:
Mainpath : up to 4k resolution, support raw/yuv format
Selfpath : up tp 1080p, support rotate, support rgb/yuv format
RSZ: scaling
DCrop: crop
ISP: 3A, Color processing, Crop
MIPI: MIPI Camera interface
Media pipelines:
Mainpath, Selfpath <-- ISP subdev <-- MIPI <-- Sensor
3A stats <-- <-- 3A parms
Code struct:
capture.c : Mainpath, Selfpath, RSZ, DCROP : capture device.
rkisp1.c : ISP : v4l2 sub-device.
isp_params.c : 3A parms : output device.
isp_stats.c : 3A stats : capture device.
mipi_dphy_sy.c : MIPI : sperated v4l2 sub-device.
Usage:
ChromiumOS:
use below v4l2-ctl command to capture frames.
v4l2-ctl --verbose -d /dev/video4 --stream-mmap=2
--stream-to=/tmp/stream.out --stream-count=60 --stream-poll
use below command to playback the video on your PC.
mplayer /tmp/stream.out -loop 0 --demuxer=rawvideo
--rawvideo=w=800:h=600:size=$((800*600*2)):format=yuy2
or
mplayer ./stream.out -loop 0 -demuxer rawvideo -rawvideo
w=800:h=600:size=$((800*600*2)):format=yuy2
Linux:
use rkcamsrc gstreamer plugin(just a modified v4l2src) to preview.
gst-launch-1.0 rkcamsrc device=/dev/video0 io-mode=4 disable-3A=true
videoconvert ! video/x-raw,format=NV12,width=640,height=480 ! kmssink
Jacob Chen (2):
media: rkisp1: add rockchip isp1 driver
ARM: dts: rockchip: add isp node for rk3288
Jeffy Chen (1):
media: rkisp1: Add user space ABI definitions
Shunqian Zheng (2):
media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
arm64: dts: rockchip: add isp0 node for rk3399
arch/arm/boot/dts/rk3288.dtsi | 24 +
arch/arm64/boot/dts/rockchip/rk3399.dtsi | 26 +
drivers/media/platform/Kconfig | 10 +
drivers/media/platform/Makefile | 1 +
drivers/media/platform/rockchip/isp1/Makefile | 9 +
drivers/media/platform/rockchip/isp1/capture.c | 1678 ++++++++++++++++++++
drivers/media/platform/rockchip/isp1/capture.h | 46 +
drivers/media/platform/rockchip/isp1/common.h | 327 ++++
drivers/media/platform/rockchip/isp1/dev.c | 728 +++++++++
drivers/media/platform/rockchip/isp1/isp_params.c | 1556 ++++++++++++++++++
drivers/media/platform/rockchip/isp1/isp_params.h | 81 +
drivers/media/platform/rockchip/isp1/isp_stats.c | 537 +++++++
drivers/media/platform/rockchip/isp1/isp_stats.h | 81 +
.../media/platform/rockchip/isp1/mipi_dphy_sy.c | 619 ++++++++
.../media/platform/rockchip/isp1/mipi_dphy_sy.h | 42 +
drivers/media/platform/rockchip/isp1/regs.c | 251 +++
drivers/media/platform/rockchip/isp1/regs.h | 1578 ++++++++++++++++++
drivers/media/platform/rockchip/isp1/rkisp1.c | 1132 +++++++++++++
drivers/media/platform/rockchip/isp1/rkisp1.h | 130 ++
drivers/media/v4l2-core/v4l2-ioctl.c | 2 +
include/uapi/linux/rkisp1-config.h | 554 +++++++
include/uapi/linux/videodev2.h | 4 +
22 files changed, 9416 insertions(+)
create mode 100644 drivers/media/platform/rockchip/isp1/Makefile
create mode 100644 drivers/media/platform/rockchip/isp1/capture.c
create mode 100644 drivers/media/platform/rockchip/isp1/capture.h
create mode 100644 drivers/media/platform/rockchip/isp1/common.h
create mode 100644 drivers/media/platform/rockchip/isp1/dev.c
create mode 100644 drivers/media/platform/rockchip/isp1/isp_params.c
create mode 100644 drivers/media/platform/rockchip/isp1/isp_params.h
create mode 100644 drivers/media/platform/rockchip/isp1/isp_stats.c
create mode 100644 drivers/media/platform/rockchip/isp1/isp_stats.h
create mode 100644 drivers/media/platform/rockchip/isp1/mipi_dphy_sy.c
create mode 100644 drivers/media/platform/rockchip/isp1/mipi_dphy_sy.h
create mode 100644 drivers/media/platform/rockchip/isp1/regs.c
create mode 100644 drivers/media/platform/rockchip/isp1/regs.h
create mode 100644 drivers/media/platform/rockchip/isp1/rkisp1.c
create mode 100644 drivers/media/platform/rockchip/isp1/rkisp1.h
create mode 100644 include/uapi/linux/rkisp1-config.h
--
2.14.2
^ permalink raw reply [flat|nested] 4+ messages in thread
* [RFC PATCH 1/5] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format
2017-11-15 7:29 [RFC PATCH 0/5] Rockchip ISP1 Driver Jacob Chen
@ 2017-11-15 7:29 ` Jacob Chen
2017-11-15 8:03 ` [RFC PATCH 0/5] Rockchip ISP1 Driver Tomasz Figa
1 sibling, 0 replies; 4+ messages in thread
From: Jacob Chen @ 2017-11-15 7:29 UTC (permalink / raw)
To: linux-rockchip
Cc: linux-media, linux-kernel, linux-arm-kernel, heiko, mchehab,
laurent.pinchart+renesas, hans.verkuil, tfiga, nicolas,
sakari.ailus, zhengsq, zyc, eddie.cai.linux, jeffy.chen,
allon.huang, p.zabel, slongerbeam, linux, Jacob Chen
From: Shunqian Zheng <zhengsq@rock-chips.com>
Add the Rockchip ISP1 specific processing parameter format
V4L2_META_FMT_RK_ISP1_PARAMS and metadata format
V4L2_META_FMT_RK_ISP1_STAT_3A for 3A.
Signed-off-by: Shunqian Zheng <zhengsq@rock-chips.com>
Signed-off-by: Jacob Chen <jacob2.chen@rock-chips.com>
---
drivers/media/v4l2-core/v4l2-ioctl.c | 2 ++
include/uapi/linux/videodev2.h | 4 ++++
2 files changed, 6 insertions(+)
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index d6587b3ec33e..0604ae9ea444 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1252,6 +1252,8 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
case V4L2_TCH_FMT_TU08: descr = "8-bit unsigned touch data"; break;
case V4L2_META_FMT_VSP1_HGO: descr = "R-Car VSP1 1-D Histogram"; break;
case V4L2_META_FMT_VSP1_HGT: descr = "R-Car VSP1 2-D Histogram"; break;
+ case V4L2_META_FMT_RK_ISP1_PARAMS: descr = "Rockchip ISP1 3A params"; break;
+ case V4L2_META_FMT_RK_ISP1_STAT_3A: descr = "Rockchip ISP1 3A statistics"; break;
default:
/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index e507b29ba1e0..14efa6513126 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -690,6 +690,10 @@ struct v4l2_pix_format {
#define V4L2_META_FMT_VSP1_HGO v4l2_fourcc('V', 'S', 'P', 'H') /* R-Car VSP1 1-D Histogram */
#define V4L2_META_FMT_VSP1_HGT v4l2_fourcc('V', 'S', 'P', 'T') /* R-Car VSP1 2-D Histogram */
+/* Vendor specific - used for IPU3 camera sub-system */
+#define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 params */
+#define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A statistics */
+
/* priv field value to indicates that subsequent fields are valid. */
#define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe
--
2.14.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [RFC PATCH 0/5] Rockchip ISP1 Driver
2017-11-15 7:29 [RFC PATCH 0/5] Rockchip ISP1 Driver Jacob Chen
2017-11-15 7:29 ` [RFC PATCH 1/5] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format Jacob Chen
@ 2017-11-15 8:03 ` Tomasz Figa
2017-11-15 14:57 ` Jacob Chen
1 sibling, 1 reply; 4+ messages in thread
From: Tomasz Figa @ 2017-11-15 8:03 UTC (permalink / raw)
To: Jacob Chen
Cc: open list:ARM/Rockchip SoC...,
Linux Media Mailing List, linux-kernel, linux-arm-kernel,
Heiko Stübner, Mauro Carvalho Chehab, Laurent Pinchart,
Hans Verkuil, nicolas, Sakari Ailus, Shunqian Zheng,
钟以崇,
Eddie Cai, Jeffy, Allon Huang, Philipp Zabel, slongerbeam, linux
Hi Jacob,
Thanks for sending the series!
On Wed, Nov 15, 2017 at 3:29 PM, Jacob Chen <jacob-chen@iotwrt.com> wrote:
> This patch series add a ISP(Camera) v4l2 driver for rockchip rk3288/rk3399 SoC.
>
> TODO:
> - Thomas is rewriting the binding code between isp, phy, sensors, i hope we could get suggestions.
> https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/768633/2
> rules:
> - There are many mipi interfaces("rx0", "dxrx0")(actually it also could be parallel interface) in SoC and isp can decide which one will be used.
> - Sometimes there will be more than one senor in a mipi phy, the sofrware should decide which one is used(media link).
> - rk3399 have two isp.
Also the two ISP subsystems have their own, completely different, CSI2
PHY blocks, so we need to make the ISP driver work with two different
PHY drivers.
> - Add a dummy buffer(dma_alloc_coherent) so drvier won't hold buffer.
> - Finish all TODO comments(mostly about hardware) in driver.
>
> To help do a quick review, i have push source code to my Github.
> https://github.com/wzyy2/linux/tree/rkisp1/drivers/media/platform/rockchip/isp1
>
> Below are some infomations about driver/hardware:
>
> Rockchip ISP1 have many Hardware Blocks(simplied):
>
> MIPI --> ISP --> DCrop(Mainpath) --> RSZ(Mainpath) --> DMA(Mainpath)
> DMA-Input --> --> DCrop(Selfpath) --> RSZ(Selfpath) --> DMA(Selfpath);)
>
> (Acutally the TRM(rk3288, isp) could be found online...... which contains a more detailed block diagrams ;-P)
>
> The funcitons of each hardware block:
>
> Mainpath : up to 4k resolution, support raw/yuv format
> Selfpath : up tp 1080p, support rotate, support rgb/yuv format
> RSZ: scaling
> DCrop: crop
> ISP: 3A, Color processing, Crop
> MIPI: MIPI Camera interface
>
> Media pipelines:
>
> Mainpath, Selfpath <-- ISP subdev <-- MIPI <-- Sensor
> 3A stats <-- <-- 3A parms
>
> Code struct:
>
> capture.c : Mainpath, Selfpath, RSZ, DCROP : capture device.
> rkisp1.c : ISP : v4l2 sub-device.
> isp_params.c : 3A parms : output device.
> isp_stats.c : 3A stats : capture device.
> mipi_dphy_sy.c : MIPI : sperated v4l2 sub-device.
>
> Usage:
> ChromiumOS:
> use below v4l2-ctl command to capture frames.
>
> v4l2-ctl --verbose -d /dev/video4 --stream-mmap=2
> --stream-to=/tmp/stream.out --stream-count=60 --stream-poll
>
> use below command to playback the video on your PC.
>
> mplayer /tmp/stream.out -loop 0 --demuxer=rawvideo
> --rawvideo=w=800:h=600:size=$((800*600*2)):format=yuy2
> or
> mplayer ./stream.out -loop 0 -demuxer rawvideo -rawvideo
> w=800:h=600:size=$((800*600*2)):format=yuy2
>
> Linux:
> use rkcamsrc gstreamer plugin(just a modified v4l2src) to preview.
>
> gst-launch-1.0 rkcamsrc device=/dev/video0 io-mode=4 disable-3A=true
> videoconvert ! video/x-raw,format=NV12,width=640,height=480 ! kmssink
Is the rkcamsrc plugin source available somewhere to download?
Thanks,
Tomasz
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [RFC PATCH 0/5] Rockchip ISP1 Driver
2017-11-15 8:03 ` [RFC PATCH 0/5] Rockchip ISP1 Driver Tomasz Figa
@ 2017-11-15 14:57 ` Jacob Chen
0 siblings, 0 replies; 4+ messages in thread
From: Jacob Chen @ 2017-11-15 14:57 UTC (permalink / raw)
To: Tomasz Figa
Cc: open list:ARM/Rockchip SoC...,
Linux Media Mailing List, linux-kernel, linux-arm-kernel,
Heiko Stübner, Mauro Carvalho Chehab, Laurent Pinchart,
Hans Verkuil, Nicolas Dufresne, Sakari Ailus, Shunqian Zheng,
钟以崇,
Eddie Cai, Jeffy, Allon Huang, Philipp Zabel, Steve Longerbeam,
linux
Hi Tomasz,
2017-11-15 16:03 GMT+08:00 Tomasz Figa <tfiga@chromium.org>:
> Hi Jacob,
>
> Thanks for sending the series!
>
> On Wed, Nov 15, 2017 at 3:29 PM, Jacob Chen <jacob-chen@iotwrt.com> wrote:
>> This patch series add a ISP(Camera) v4l2 driver for rockchip rk3288/rk3399 SoC.
>>
>> TODO:
>> - Thomas is rewriting the binding code between isp, phy, sensors, i hope we could get suggestions.
>> https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/768633/2
>> rules:
>> - There are many mipi interfaces("rx0", "dxrx0")(actually it also could be parallel interface) in SoC and isp can decide which one will be used.
>> - Sometimes there will be more than one senor in a mipi phy, the sofrware should decide which one is used(media link).
>> - rk3399 have two isp.
>
> Also the two ISP subsystems have their own, completely different, CSI2
> PHY blocks, so we need to make the ISP driver work with two different
> PHY drivers.
>
>> - Add a dummy buffer(dma_alloc_coherent) so drvier won't hold buffer.
>> - Finish all TODO comments(mostly about hardware) in driver.
>>
>> To help do a quick review, i have push source code to my Github.
>> https://github.com/wzyy2/linux/tree/rkisp1/drivers/media/platform/rockchip/isp1
>>
>> Below are some infomations about driver/hardware:
>>
>> Rockchip ISP1 have many Hardware Blocks(simplied):
>>
>> MIPI --> ISP --> DCrop(Mainpath) --> RSZ(Mainpath) --> DMA(Mainpath)
>> DMA-Input --> --> DCrop(Selfpath) --> RSZ(Selfpath) --> DMA(Selfpath);)
>>
>> (Acutally the TRM(rk3288, isp) could be found online...... which contains a more detailed block diagrams ;-P)
>>
>> The funcitons of each hardware block:
>>
>> Mainpath : up to 4k resolution, support raw/yuv format
>> Selfpath : up tp 1080p, support rotate, support rgb/yuv format
>> RSZ: scaling
>> DCrop: crop
>> ISP: 3A, Color processing, Crop
>> MIPI: MIPI Camera interface
>>
>> Media pipelines:
>>
>> Mainpath, Selfpath <-- ISP subdev <-- MIPI <-- Sensor
>> 3A stats <-- <-- 3A parms
>>
>> Code struct:
>>
>> capture.c : Mainpath, Selfpath, RSZ, DCROP : capture device.
>> rkisp1.c : ISP : v4l2 sub-device.
>> isp_params.c : 3A parms : output device.
>> isp_stats.c : 3A stats : capture device.
>> mipi_dphy_sy.c : MIPI : sperated v4l2 sub-device.
>>
>> Usage:
>> ChromiumOS:
>> use below v4l2-ctl command to capture frames.
>>
>> v4l2-ctl --verbose -d /dev/video4 --stream-mmap=2
>> --stream-to=/tmp/stream.out --stream-count=60 --stream-poll
>>
>> use below command to playback the video on your PC.
>>
>> mplayer /tmp/stream.out -loop 0 --demuxer=rawvideo
>> --rawvideo=w=800:h=600:size=$((800*600*2)):format=yuy2
>> or
>> mplayer ./stream.out -loop 0 -demuxer rawvideo -rawvideo
>> w=800:h=600:size=$((800*600*2)):format=yuy2
>>
>> Linux:
>> use rkcamsrc gstreamer plugin(just a modified v4l2src) to preview.
>>
>> gst-launch-1.0 rkcamsrc device=/dev/video0 io-mode=4 disable-3A=true
>> videoconvert ! video/x-raw,format=NV12,width=640,height=480 ! kmssink
>
> Is the rkcamsrc plugin source available somewhere to download?
It could be download, but it's rough and just be used by me to test
driver at present since
Hal3 is not completed.
You can consider it as "media-ctl + gstreamer v4l2src"
https://github.com/GStreamer/gst-plugins-good/blob/master/sys/v4l2/gstv4l2src.c
>
> Thanks,
> Tomasz
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2017-11-15 14:58 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-11-15 7:29 [RFC PATCH 0/5] Rockchip ISP1 Driver Jacob Chen
2017-11-15 7:29 ` [RFC PATCH 1/5] media: videodev2.h, v4l2-ioctl: add rkisp1 meta buffer format Jacob Chen
2017-11-15 8:03 ` [RFC PATCH 0/5] Rockchip ISP1 Driver Tomasz Figa
2017-11-15 14:57 ` Jacob Chen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).